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PREFACE TO THE 
SECOND EDITION 


Because the first edition of this book was well received by the academic and 
engineering community, a special attempt was made in the second edition to 
include only those changes that seemed to clearly improve the book's use in the 
classroom. Most of the modifications were included only after obtaining input 
from several users of the book. 

Except for a few minor corrections and additions, just six significant changes 
were made. Only two, a new section on the central limit theorem and one on 
gaussian random processes, represent modification of the original text, A third 
change, a new chapter (10) added at the end of the book, serves to illustrate a 
number of the book’s theoretical principles by applying them to: problems 
encountered in practice. A fourth change is the addition of Appendix F, which is 
a convenient list of some useful probability densities that are often encountered. 

The remaining .two changes are probably the most significant, especially for 
instructors using the book. First, the number of examples that illustrate the 
topics discussed has been increased by about 30 percent (over 85 examples are 
now included), These examples were carefully scattered throughout the text in an 
effort to include at least one in each section where practical to do so. Second, 
over 220 new student exercises (problems) have been added at the ends of the 
chapters (a 54 percent increase). 

The book now contains 630 problems and a complete solutions manual is 
available to instructors from the publisher. This addition was in response to in- 
structors that had used most of the exercises in the first edition. For these instruc- 
tors’ convenience in identifying the new problems, they are listed in each chapter 
as “Additional Problems.” 
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All other aspects of the book, such as ils purpose (a textbook), intended 
audience (juniors, seniors, first-year graduate students), level, and style of presen- 
tation, remain as before. 

I would like to thank D, I. Starry for her excellent work in typing the manu- 
script and the University of Florida for making her services available. Finally, | 
am again indebted to my wife, Barbara, for her selfless efforts in helping me 


proofread the book. If the number of in-print errors is small, it is greatly due to 
her work. 


Peyton Z. Peebles, Jr. 


PREFACE TO THE 
FIRST EDITION 


This book has been written specifically as a textbook with the, purpose of intro- 
ducing the principles of probability, random variables, and random signals to 
either junior or senior engineering students, 

The level of material included in the book has been selected to apply to a 
typical undergraduate program, However, a small amount of more advanced 
material is scattered throughout to serve as stimulation for the more advanced 
student, or to fill out course content in schools where students are at a more 
advanced level. (Such topics are keyed by a star *.) The amount of material 
included has been determined by my desire to fit the text to courses of up to one 
semester in length. (More is said below about course structure.) 

The need for the book is easily established. The engineering applications of 
probability concepts have historically been taught at the graduate level, and 
many excellent texts exist at that level. In recent times, however, many colleges 
and universities are introducing these concepts into the undergraduate curricula, 
especially in electrical engineering. This fact is made possible, in part, by refine- 
ments and simplifications in the theory such that it can now be grasped by junior 
or senior engineering students. Thus, there is a definite need for a text that is 
clearly written in a manner appealing to such students. I have tried to respond to 
this need by paying careful attention to the organization of the contents, the 
development of discussions in simple language, and the inclusion of text examples 
and many problems at the end of each chapter. The book contains over 400 
problems and a solutions manual for all problems is available to instructors from 
the publisher, 

Many of the examples and problems have purposely been made very simple 
in an effort to instill a sense of accomplishment in the student, which, hopefully, 
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will provide the encouragement to go on to the more challenging problems. 
Although emphasis is placed on examples and problems of electrical engineering, 
the concepts and theory are applicable to all areas of cnginccring. 

The International System of Units (SI) has been used primarily throughout 
the text. However, because technology is presently in a transitional stage with 
regard to measurements, some of the more established customary units (gallons, 
°F, ete) are also utilized; in such instances, values in SI units follow in paren- 
theses. : 

The student background required to study the book is only that typical of 
junior or senior engincering students. Specifically, it is assumed the student has 
been introduced to multivariable calculus, Fourier series, Fourier transforms, 
impulse functions, and some linear system theory (transfer function concepis, 
especially), 1 recognize, however, that students tend to forget a fair amount of 
what is initially taught in many of these areas, primarily through lack of 
opportunity to apply the material in later courses. Therefore, 1 have inserted 
short reviews of some of these required topics. These reviews are occasionally 
included in the text, but, for the most part, exist in appendixes at the end of the 
book. 

The order of the material is dictated by the main topic. Chapter | introduces 
probability from the axiomatic definition using set theory. In my opinion this 
approach is more modern and mathematically correct than other definitions. It 
also has the advantage of creating a better base for students desiring to go on to 
graduate work, Chaplec 2 introduces the theory of a single candom variable, 
Chapter 3 introduces operations on one random variable that are based on sta- 
tistical expectation, Chapter 4 extends the theory to several random variables, 
while Chapter 5 defines operations with several variables. Chapters 6 and 7 intro- 
duce random processes. Definitions based on temporal characterizations arc 
developed in Chapter 6, Spectral characterizations are included in Chapter 7. 

The remainder of the text is concerned with the response of linear systems 
with random inpuls, Chapler & contains the general theory, mainly for linear 
time-invariant systems; while Chapter 9 considers specific optimum systems that 
either maximize system outpul signal-to-noise ratio or minimize a suitably 
defined average error. 

Finally, the book closes with a number of appendixes that contain material 
helpful to the student in working problems, in reviewing background topics, and 
in the interpretation of the text. 

The book can profitably be used in curricula based on cither the quarter or 
the semester system. At the University of Tennessee, one-quarter undergraduate 
course at the junior level has been successfully taught that covers Chapters 1 
through 8, except for omitting Sections 2.6, 3.4, 4.4, 8.7 through 8.9, and all 
starred material. The class met three hours per week. 

A one-semester undergraduate course (three hours per week) can readily be 
structured to cover Chapters | through 9, omitting all starred material except 
that in Sections 3.3, 5.3, 7.4, and 8.6. 

Although the text is mainly developed for the undergraduate, | have also 
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successfully used it in a one-quarter graduate course (first-year, three hours per 
week) that covers Chapters | through 7, including all starred material. 

It should be possible to cover the entire book, including all starred material, 
in a one-semester graduate course (first-year, three hours per week). 

Lam indebted to many people who have helped make the book possible. Drs. 
R. C. Gonzalez and M. O. Pace read portions of the manuscript and suggested a 
number of improvements. Dr. T. V. Blalock taught from an early version of the 
manuscript, independently worked'a number of the problems, and provided 
various improvements. I also extend my appreciation to the Advanced Book 
Program of Addison-Wesley Publishing Company for allowing me to adapt and 
use several of the figures from my earlier book Communication System Principles 
(1976), and to Dr. J. M. Googe, head of the electrical engineering department of 
the University of Tennessce, for his support and encouragement of this project. 
Typing of the bulk of the manuscript was ably done by Ms. Belinda Hudgens, 
other portions and various corrections were typed by Kymberly Scott, Sandra 
Wilson, and Denise Smiddy. Finally, f thank my wife, Barbara, for her aid in 
proofreading the entire book. 


Peyton Z. Peebles, Jr. 
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10 INTRODUCTION TO BOOK AND CHAPTER 


The primary goals of this book are to introduce the reader to the principles of 
random signals and to provide tools whereby one can deal with systems involv- 
ing such signals. Toward these goals, perhaps the first (hing that should be donc 
is define what is meant by random signal. A random signal is a time waveformt 
that can be characterized only in some probabilistic manner, In general, it can be 
either a desired or undesired waveform. 

The reader has no doubt heard background hiss while listening to an ordi- 
nary broadcast radio receiver. The waveform causing the hiss, when observed on 
an oscilloscope, would appear as a randomly fluctuating voltage with time, It is 
undesirable, since it interferes with our ability to hear the radio program, and is 
called noise. 

Undesired random waveforms (noise) also appear in the outputs of other 
types of systems. In a radio astronomer’s receiver, noise interferes with the 
desired signal from outer space (which itself is a random, bul desirable, signal). In 
a television system, noise shows up in the form of picture interference often called 
“snow.” In a sonar system, randomly generated sea sounds give rise to a noise 
that interferes with the desired echoes. : 

The number of desirable random signals is almost limitless. For example, the 
bits in a computer bit stream appear to fluctuate randomly with time between the 


t We shall usually assume random signals to be voltage-time waveforms. However, the theory to 
be developed throughout the book will apply, in most cases, to random functions other than voltage, 
of arguments other than time. 
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zero and one states, thereby creating a random signal. In another example, the 
output voltage of a wind-powered generator would be random because wind 
specd Muctuates randomly. Similarly, the voltage from a solar detector varics ran- 
domly due to the randomness of cloud and weather conditions. Still other cxam- 
ples are: the signal from an instrument designed to measure instantancous ocean 
wave height; the space-originaled signal at the output of the radio astronomer's 
antenna (the relative intensity of this signal from space allows the astronomer to 
form radio maps of the heavens); and the voltage from a vibration analyzer 
attached lo an automobile driving over rough terrain, 

In Chapters 8 and 9 we shall study methods of characterizing systems having 
random input signals. However, from the above examples, it is obvious that 
random signals only represent the behavior of more fundamental underlying 
random phenomena. Phenomena associated with the desired signals of the last 
paragraph are: information source for computer bit stream; wind speed; various 
weather conditions such as cloud density and size, cloud speed, elc.; occan wave 
height; sources of ouler spice signals; and terrain roughness. All these phenom- 
ena must be described in some probabilistic way. 

Thus, there are actually two things to be considered in characterizing 
random signals. Onc is how to describe any one of a variety of random phenom- 
ena; another is how to bring time into the problem so as to create the random 
signal of interest. To accomplish the first item, we shall introduce mathematical 
concepts in Chapters 2, 3, 4, and 5 (random variables) that are sufficiently general 
they can apply to any suitably defined random phenomena. To accomplish the 
second item, we shall introduce another mathematical concept, called a random 
process, in Chapters 6 and 7, All these concepts are based on probability theory. 

The purpose of this chapter is to introduce the elementary aspects of prob- 
ability theory on which all of our later work is based. Several approaches exist 
far the definition and discussion of probability. Only two of these are worthy of 
modern-day consideration, while all others are mainly of historical interest and 
are not commented on further here. Of the more modern approaches, one uses 
the relative frequency definition of probability. It gives a degree of physical 
insight which is popular with engineers, and is often used in texts having prin- 
cipal topics other than probability theory itself (for example, see Pecbles, 1976).t 

The second approach to probability uses the axiomatic definition. It is the 
most mathematically sound of all approaches and is most appropriate for a text 
having its lopics based principally on probability theory. The axiomatic 
approach also serves as the best basis for readers wishing to proceed beyond the 
scope of this book to more advanced theory. Because of these facts, we adopt the 
axiomatic approach in (his book. 

Prior to the introduction of the axioms of probability, it is necessary that we 
first develop certain elements of set theory. 


t References are quoted by name and date of publication. They are listed at the end of the hook, 
t Our treatment is limited to the level required to introduce the desired probability concepts. For 
additional details the reader is referred lo MéFadden (1963), or Millon and Tsokos (1976). 
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1.1 SET DEFINITIONS 


A set is a collection of objects. The objects are called elements of the set and may 

. be anything whatsoever. We may have a set of voltages, a set of airplanes, a set of 
chairs, or even a set of sets, called a class of scts. A set is usually denoted by a 
capital letter while an element is represented by a lower-case letter. Thus, if a is 
an clement of set A, then we write 


aéAdA 


(1.1-1) 


Ifa is not an element of A, we write 


agA (1.1-2) 


A set is specified by the content of two braces: {-}. Two methods exist for 
specifying content, the tabular method and the rule method. In the tabular 
method the elements are enumerated explicitly. For example, the set of all in- 


_ tegers between 5 and 10 would be {6, 7, 8, 9}. In the rule method, a set’s content 


is determined by some rule, such as: {integers between 5 and 10}.t The rule 
method is usually more convenient to use when the set is large. For example, 
{integers from 1 to 1000 inclusive} would be cumbersome to write explictly using 
the tabular method. 

A set is said to be countable if its elements can be put in one-to-one corre- 
spondence with the natural numbers, which are the integers 1, 2, 3, etc. Ifa set is 
not countable it is called uncountable. A set is said to be empty if it has no ele- 
ments. The empty set is given the symbol @ and is often called the null set. 

A finite set is one that is cither empty or has elements that can be counted, 
with the counting process terminating. In other words, it has a finite number of 


_ clements. If a set is not finite it is called infinite. An infinite set having countable 


elements is called countably infinite. 
If every element of a set A is also an element in another set B, A is said to be 
contained in B. A is known as a subset of B and we write 


‘ ASB (1.t-3) 
If at least one element exists in B which is not in A, then A is a proper subset of B, 
denoted by (Thomas, 1969) 
Ac B (1.1-4) 
The null set is clearly a subset of all other scts. 
Two sets, A and B, are called disjoint or mutually exclusive if they have no 
common clements. 
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t Sometimes notations such as (1[5 << 10,7 an integer} or (1:5 <1 < 10, fan integer} are 
seen in the literature. 
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4 PROBABILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 


Example 1.1-1 To illustrate the topics discussed above, we identify the sets 


listed below. 


A= (1,3, 5,7} D = {0.0} 
B= {1,2,3,...} E = {2, 4, 6, 8, 10, 12, 14} 
C= (05 <c< 8.5} F = {-5.0 <f< 12.0} 


The set A is tabularly:specified, countable, and finite. B is also tabularly 
specified and countable, but is infinite. Set C is rule-specified, uncountable, 
and infinite, since it contains all numbers greater than 0.5 but not exceeding 
8.5. Similarly, sets D and E are countably finite, while set F is uncountably 
infinite. It should be noted that D is not the null set; it has one element, the 
number zero. ' 

Set A is contained in sets B, C, and F. Similarly, Co F, Dc F, and 
Ec B, Sets B and F are not subsets of any of the other sets or of each other. 
Sets A, D, and E are mutually exclusive of each other, The reader may wish 
to identify which of the remaining sets are also mutually exclusive. 


The largest or all-encompassing set of objects under discussion in a given 
situation is called the wniversal set, denoted S. All sets (of the situation 
considered) are subsets of the universal set. An example will help clarify the 
concept of a universal set. 


Example 1,1-2 Suppose we consider the problem of rolling a die. We are 
interested in the numbers that show on the upper face. Here the universal set 
is S= {1, 2, 3, 4, 5, 6}. In a gambling game, suppose a person wins if the 
number comes up odd, This person wins for any number in the set A = 


{1, 3, 5}. Another person might win if the number shows four or less; that is, 


for any number in the set B = {1, 2, 3, 4}. 
Observe that both A and 8 are subsets of S. For any universal set with N 


elements, there are 2% possible subsets of S. (The reader should check this for - 


a few values of N.) For the present example, N = 6 and 2” = 64, so that there 
are 64 ways one can define “ winning” with one die. 


It should be noted that winning or losing in the above gambling game is 
related to a set. The game itself is partially specified by its universal set (other 
games typically have a different universal set). These facts are not just coin- 
cidence, and we shall shortly find that sets form the basis on which our study of 
probability is constructed. 
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Figure 1.2-1 Venn diagrams. (a) Illustration 
of subsets and mutually exclusive sets, and 
(b) illustration of intersection and union of 
sets. (Adapted from Peebles,(1976) with permis- 
sion of publishers Addison-Wesley, Advanced 
Book Program.) 


1.2 SET OPERATIONS 


In working with sets, it is helpful to introduce a geometrical representation that 
enables us to associate a physical picture with sets. 


Venn Diagram 


Such a representation is the Venn diagram.t Here sets are represented by closed- 
plane figures, Elements of the sets are represented by the enclosed points (area). 
Thé universal set S is represented by a rectangle as illustrated in Figure 1.2-la. 
Three sets A, B, and C are shown, Set C is disjoint from both A and B, while set 
B is a subset of A. 


Equality and Difference 


Two sets A and B are equal if all elements in A are present in B and all elements 
in Bare present in A; that is, if A S Band B ¢ A. For cqual sets we write A = B. 
The difference of two sets A and B, denoted A — B, is the set containing all 


t After John Venn (£834-1923), an Englishman. 
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clements of A that are not present in B. For example, with A = {0.6<as 1.6} 
and B= {0 <b < 2.5}, then A—B={06<c< 1.0} or B-A= {l6<ds 
2.5}. Note that A — B# B— A. 


Union and Intersection 
The union (call it C) of (wo sets A and B is written 
C=AUB 


It is the set of all elements of A or B or both. The union is sometimes called the 


stun of two sets. 
The intersection (call it D) of two sets A and B is written 


D=ANB (1.2-2) 


It is the set of all elements common to both A and B. Intersection is sometimes 
called the product of two sets. For mutually exclusive sets A and B, AN B=@. 
Figure 1.2-1 illustrates the Venn diagram area to be associated with the intersec- 


tion and union of sets. 
By repeated application of (1.21) or (1.2-2), the union and intersection of N 
scts A,," = 1, 2,..., N, become 


CHA, UA, Us UAN= UA, (1.2-3) 


D=A, nA, 01° A An= (4p (1.2-4) 


Complement 

The complement of a set A, denoted by A, is the set of all elements not in A. Thus, 
Az=S-A (1.2-5) 

It is also casy to sce that @=S,5=G,Av Az=S,andANA=@. 


Exaynple 1.2-1 We illustrate intersection, union, and complement by taking 
an example with (he four sets 


B = (2, 6,7, 8, 9, 10, It} 
C = (1,3, 4, 6 7, 8} 


y= {1 < integers s 12} 
A ={1,3, 5, 12} 
Applicable unions and intersections here are: 
Au B= (1,2, 3,5, 6 7,89 10, 11, 12} 
Avc=({1,34,5,6 7 8 12} 
BUC=(1,2,3, 4, 6, 7, 8, 9, 10, 11} 


ANB=D 
Ance#=({li,3} 
BoC = {6, 7, 8} 


(12-1) 


vr 
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Figure 1.2-2 Venn diagram applicable to 
Example !.2-1. 


Complements are: 
A = (2,4, 6, 7, 8, 9, 10, 11} 


B = (1,3, 4, 5, 12} 
C = (2, 5,9, 10, 11, 12} 


The various sets are illustrated in Figure 1.2-2. 


Algebra of Sets 


All subsets of the universal set form an algebraic system for which a number of 
theorems may be stated (Thomas, 1969). Three of the most important of these 
relate to laws involving unions and intersections. The commutative law states that 


ANB=BOA (1.2-6) 
AUB=BUA (1.2-7) 

| The distributive law is written as 
. AN(BUQS(AN BY (AnC) (1.2-8) 
AV(BAC (AY B)A(AV C) (1.2-9) 

The associative law is wrilten as 
(AV B)UCHAU(BUO)=AUBUC (1.2-10) 
(AN B)ACHAN(BAOC)FANBNC (1.2-11) 


These are just restatements of (1.2-3) and (1.2-4). 


De Morgan’s Laws 


By use of a Venn diagram we may readily prove De Morgan's lawst, which state 
that the complement of a union (intersection) of two sels A and B equals the 
intersection (union) of the complements A and B. Thus, 


(1.2-12) 
(1.2-13) 


ANB 
AUB 


t After Augustus De Morgan (1806-1871), an English mathematician. 
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From the last two expressions one can show that if in an identity we replace. 


unions by intersections, intersections by unions, and sets by their complements, 
then the identity is preserved (Papoulis, 1965, p. 23). 


Example 1.2-2 We verify De Morgan's law (1.2-13) by using the example sets 
A={2<a< 16} and B= {5 <b < 22} when S = {2 <s ¢ 24}. First, if we 
define C=Anq B, the reader can readily see from Venn diagrams that 
C=ANB={5<cs 16} so C= AN B= {(2<c5, 16<c < 24}. This 
result is the left side of (1.2-13). 

Second, we compute A=S—A={16<a<24} and B=S~—B= 
{2<b<5, 22<bs 24}, Thu, C=AUB=(2<c55, 16<c< 24}, 
This result is the right side of (1.2-13) and De Morgan’s law is verified. 


Duality Principle 


This principle (Papoulis, 1965) states: ifin an identity we replace unions by inler- 
sections, intersections by unions, S by @, and @ by S, then the identity is pre- 
served. For example, since 


AN(BUC)=(ANB)U(ANC) (1.2-14) 
is a valid identity from (1.2-8), it follows that 
AU(BNC)=(A VU B)A(AUC) (1.2-15) 


is also valid, which is just (1.2-9). 


13 PROBABILITY INTRODUCED THROUGH SETS 


Basic to our study of probability is the idea of a physical experiment. In this 
section we develop a mathematical model of an experiment. Of course, we are 
interested only in experiments that are regulated in some probabilistic way. A 


single performance of the experiment is called a trial for which there is an 
outcome. 


Experiments and Sample Spaces 


Although there exists a precise mathematical procedure for defining an experi-. 
ment, we shall rely on reason and examples. This simplified approach will ulti- 
mately lead us to a valid mathematical model for any real experiment.t To 


t Most of our early definitions involving probability are rigorously established only through con- 
cepis beyond our scope. Although we adopt u simplified development of the theory, our final results 
are no less valid or useful than if we had used the advanced concepts. 


rronanmnrry 9 


illustrate, one experiment might consist of rolling a single die and observing the 
number that shows up. There are six such numbers and they form all the possible 
outcomes in the experiment. If the die is “unbiased” our intuition (ells us thal 
cach outcome is equally likely to occur and the likelihaod of any one occurring is 
YF (later we call this number the probability of the outcome). This experiment is 
seen (o be governed, in part, by (wo sets. One is the set of all possible outcomes, 
and the other is the set of the likelihoods of the outcomes, Each set has six cle- 
ments, For the present, we consider only the set of outcomes. 

The set of all possible outcomes in any given experiment is called the sample 
space and it is given the symbol S, In effect, the sample space is a universal set for 
the given experiment. S may be different for different experiments, but all experi- 
ments are governed by some sample space. The definition of sample space forms 
the first of three clements in our mathematical model of experiments. The remain- 
ing clements are events and probability, as discussed below. 


Discrete and Continuous Sample Spaces 


In the carlier dic-tossing experiment, S was a finite set with six clements. Such 
sample spaces are said to be discrete and finite, The sample space can also be dis- 
crete and infinite for some experiments. For example, § in the experiment 
“choose randomly a positive integer” is (he countably infinite set ae Paes 

Some experiments have an uncountably infinite sample space, An i!lustration 
would be the experiment “obtain a number by spinning the pointer on a wheel of 
chance numbered from 0 to 12." Here any number s from 0 to 12 can result and 
S = {0 <s < 12}. Such a sample space is called continuous. 


Events 


In most situations, we are interested in some characteristic of the oulcomes of our 
experiment as opposed to the outcomes themselves, In the experiment “draw a 
card from a deck of 52 cards,” we might be more interested in whether we draw a 
spade as opposed to having any interest in individual cards. To handle such situ- 
ations we define the concept of an event. 

An event is defined as a subset of the sample space. Because an event is a set, 
all the earlier definitions and operations applicable to sets will apply to events. 
For example, if two events have no common outcomes they are mutually 
exclusive. 

In the above card experiment, 13 of the 52 possible outcomes are spades. 
Since any one of the spade outcomes satisfies the event “draw a spade,” this 
event is a set with 13 clements. We have earlier stated that a set with N elements 
can have as many as 2% subsets (events defined on a sample space having N 
possible outcomes), In the present example, 2% = 2°? ~ 4.5(10'*) events. 

As with the sample space, events may be either discrete or continuous, The 
card event “draw a spade” is a discrete, finite event. An example of a discrete, 
countably infinite event would be “select an odd integer” in the experiment 
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“randomly select a positive integer.” The event has a countably infinite number 
of clements: {1, 3, 5, 7, «-.}. However, events defined on a coutably infinite sample 
space do not have to be countably infinite. The event {1, 3, 5, 7} is clearly not 
infinite but applies to the integer sclection experiment. 

Events defined on continuous sample spaces arc usually continuous. In the 
experiment “choose randomly a number a from 6 to 13,” the sample space is 
S=({6<s< 13}. An event of interest might correspond to the chosen number 
falling between 7.4 and 7.6; that is, the event (call it A) is A = (14<a< 7.6}. 

Discrete events may also be defined on continuous sample spaccs. An 
example of such an event is A = {6.13692} for the sample space S = (6558 13} 
of the previous paragraph, We comment later on this type of event. 

The above definition of an event as a subset of the sample space forms the 
second of three elements in our mathematical model of experiments. The third 
element involves defining probability. 


Probability Definition and Axioms 


To cach event defined on a sample space S, we shall assign a nonnegative number 
called probability. Probability is therefore a function; it is a function of the events 
defined. We adopt the notation P(A)t for “the probability of event A.” When an 
event is stated explicitly as a set by using braces, we employ the notation P{-} 
instead of P({+}). 

The assigned probabilitics are chosen so as to salisfy three axioms. Let A be 
any event defined ona sample space S. Then the first two axioms are 


P(A) 20 
P(S) = 1 


(1.3-1a) 
(1.3-1h) 


axiom 1: 
axiom 2: 


The first only represents our desire to work with nonnegative numbers. The 
second axiom recognizes that the sample space itself is an event, and, since it is 
the all encompassing event, it should have the highest possible probability, which 
is selected as unity. For this reason, S is known as the certain event. Alternatively, 
the null set @ is an event with no elements; it is known as the impossible event 
and its probability is 0. 

The third axiom applies to Nevents Ay, = 1,200 N, where N may possi- 
bly be infinite, defined on a sample space S, and having the property 4, 9 A, = 
@ forall in # a Ut is 


N N 
axiom 3: rf U A,) = >, P(A,) if 


nal amt 


Am 0 A, = BD (1.3-1¢) 


for alam An Hl, 2, N, with N possibly infinite. The axiom states that the 


t Oceasionally it will be convenient to use brackets, such as PLA when A is itself an event such as 
CH-(Ba lt. 


k 
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probability of the event equatlo the union of any number of mutually exclusive 
events is equal to the sum of tle individual event probabilitics. 

An example should help jive a physical pictire of the meaning of the above 
axioms. 


a ee rrr atte 
Example 1.3-1 Let an eipcriment consist of obtaining a number x by spin- 
ning the pointer on a “fair” wheel-of chirce that is labeled from 0 to 100 
points. The sample spat isS={O<xS 1@)}. We reason that probability of 
the pointer falling betwien any (wo numbers X2 2X1 should be (x2 — x,)/100 
since the wheel is fair.As a check on this assignment, we sce that the event 
A=z{x,<xs x} satisfies axiom 1 for all x, and x,, and axiom 2 when 
x, = 100 and x, = 0. 

Now suppose we break the wheel's periphery into N contiguous scg- 
ments A, = {Xa-1 <* S Xa} Xn = (n)OO/N, w= 1, 2 oe N, with xo = 0. 
Then P(A,) = 1/N, and, for any N, 


N N N 4 
rf UA) = ¥ PA) = Lys! = P(S) 
nel 


awl asl 
from axiom 3. 


Example 1.3-1 allows us to return to our earlier discussion of discrete events 
defined on continuous sample spaces. If the interval x, — X,-1 is allowed to 
approach zero (— 0), the probability P(A,)—> P(Xn)s that is, P(A,) becomes the 
probability of the pointer falling exactly on the point x,. Since N— 00 in this 
situation, P(A,)— 0. Thus, the probability of a discrete event defined on a contin- 
uous sample space is 0. This fact is true in general. 

A consequence of the above statement is that events can occur even if their 
probability is 0. Intuitively, any number can be obtained from the whecl of 
chance, but that precise number may never occur again. The infinite sample space 
has only one outcome satisfying such a discrete event, so its probability is 0. Such 
events are not the sume as the impossible event which has no clements and cannot 
occur. The converse situation can also happen where events with probability | 
may not occur. An‘example for the wheel of chance experiment would be the 
event A = {all numbers except the number x,}. Events with probability 1 {that 
may not occur) are not the same as the certain event which must occur, 


Mathematical Model of Experiments 


The axioms of probability, introduced above, complete our mathematical model 
of an experiment. We pause to summarize. Given some real physical experiment 
having a set of particular outcomes possible, we first defined a sample space to 
mathematically represent the physical oulcomes. Second, it was recognized that 
certain characteristics of the outcomes in the real experiment were of interest, as 
oppused to the outcomes themselves; events were defined to mathematically 
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om 


i: ts iano. these characteristics. Finally, probablities were assigned to the defined As a matter of interest, we also observe the probabilitics of the events 
e events to mathematically account for the randon nature of the experiment. BAC and BUC to be P(B A C)= Aho) =e and P(BUC)= 
Mf i Thus, a real experiment is defined mathemaically by three things: (1) assign- 10(‘A6) = J, 

Be ment of a sample space; (2) definition of events of interest; and (3) making prob- 0 aes Sacte e 
| i . ee the events such that the acioms are satisfied. Establishing 

Ne rect model for an exptriment is probably the single most difficult step in 


solving probability problems. 1.4 JOINT AND CONDITIONAL PROBABILITY 


% 


> omny; 


ws 
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Example 1.3-2 An experiment consists of observing the sum of the numbers 
showing up when two dice are thrown. We: develop a model for this 
experiment. ‘ 

The sample space consists of 6? = 36 points as shown in Figure {.3-1. 
Each possible outcome corresponds to a sum having values from 2 to 12. 

Suppose we are mainly interested in three events defined by A= 
{sum = 7}, B={8<sum< 11}, and C= {10 <sum}. ‘In assigning proba- 
bilities to these events, it is first convenient to define 36 elementary events 
A, = {sum for outcome (i, j)=i+j}, where i represents the row and j repre- 
sents the column locating a particular possible outcome in Figure 1.3-1, An 
elementary event has only one element. . 

For probability assignments, intuition indicates that each possible out- 
come has the same likelihood of occurrence if the dice are fair, so P(A,)) = 
56: Now because the evenis A,, i and j= 1, 2,..., N= 6, are mutually 
exclusive, they must satisfy axiom 3. But since the events A, B, and C are 
simply the unions of appropriate elementary events, their probabilities are 
derived from axiom 3, From Figure 1.3-1 we easily find 


, ‘ 
P(A) = Uv) = 2 PAL 1-) = (%) =t 


In some experiments, such as in Example 1.3-2 above, it may be that some events 
are not mutually exclusive because of common elements in the sample space. 
These elements correspond to the simultaneous or joint occurrence of the non- 
exclusive events. For two events A and B, the common elements from the event 
ANB. 


Joint Probability 


The probability P(A © B) is called the joint probability for two events A and B 
which intersect in the sample space. A study of a Venn diagram will readily show 
that 


P(A cB) = P(A) + P(B) — P(A vv B) (1.4-1) 

Equivalently, 
P(A U B) = P(A) + P(B) — P(A 7 B) Ss P(A) + P(B) (1.4-2) 
In other words, the probability of the union of two events never exceeds the sum 


of the event probabilities. The equality holds only for mutually exclusive events 
because A A B = @, and therefore, P(A 7 B) = P(G) = 0, 


Conditional Probability 


Given some event B with nonzero probability 


i 
P(C) = (=) me P(B) > 0 (1.4-3) 
: 36 12 we deline the conditional probability of an event A, given B, by 
P(A co B) 
(Al 8) ==> 4-4 
P(A|B) P(B) (1.4-4) 


The probability P(A |B) simply reflects the fact that the probability of an event A 
may depend on a sccond event B. If A and Bare mutually exclusive, A a B= SO, 
and P(A |B) = 0. : 

Conditional probability is a defined quantity and cannot be proven. 
However, as a probabilily it must satisfy (he three axioms given in (1.3-1). P(A | B) 
obviously satisfies axiom | by its definition because P(A m B) and P(B) are non- 
negative numbers. The second axiom is shown to be satisfied by letting S = A: 


pn op ae ca 


(5, 4) (5, 5) 


(6, 4) 


(eR8) ba 


= 


P(S| B) = se (1.4-5) 


Figure 1.3-1 Sample space upplicuble to Example 1.3-2. P(B) P(B) . 
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Table 1.4-1 Numbers of resistors 


! 
The third axiom may be shown to hold by considering the union of A with an : ation ‘ 
in a box having given resistance and 


event C, where A and C are mutually exclusive. If P(A vu C|B) = P(A| B) + 
P(C| B) is true, then axiom 3 holds. Since A A C = @ then events Am B and tolerance. 
Bo Care mutually exclusive (use a Venn diagram lo verify this fact) and Tolerance 
PU(A uC) 0 B] = PU(A 9 B) (C9 BY = P(A 9B) + PIC B) Resistance (9) 5% 10% __ Total i 
(1.4-6) 2 10 14 24 : 
ee 47 28 16 44 ae 
Thus, on substitution into (1.4-4) 100 24 8 32 Hl 
‘wit 
PAU C)A BB] P(A AB) (CAB Total 6238 10 fe 
P((A u C)|B] = aco a ne) noe!) Oe ei 
P(B) P(B) P(B) ae 
as) 
= P(A|B) + P(C|B) (1.4-7) By using (1.4-4) the conditional probabilities become a 
and axiom 3 holds. P(A 7 B) 28 
P(A|B) = —— 
ale) P(B) 62 
op Set pee 
P(A AC) 
Example 1.4-1 In a box there are 100 resistors having resistance and toler- P(A|C) = “PO 0 
ance as shown in Table 1.4-1. Let a resistor be selected from the box and . : 
assume each resistor has the same likelihood of being chosen. Define three - PIBAC)_ 24 
P(B\C) =—FC) =33 


events: A as “draw a 47-2 resistor,” B as “draw a resistor with 5% toler- 
ance,” and C as “draw a 100-Q resistor.” From the table, the applicable 


probabilities arct 


44 
P(A) = P(47 2) = 75 


62 
P(B) = P(5%) = 00 


P(C) = P(100 Q) = = 

The joint probabilities are 
P(A nm B) = PAT. 5%) = oe 
100 
P(A a C) = P(47 2.4 100 Q) = 0 


24 
= P(5S% Y= —— 
P(B A C) = P(S% A 100 9) = 75 


abilities are related (o the number of resistors in the box that salisfy an 
be selected. An alternative approach would be based on 
ample J.3-2. The reader may view the latter approach as 


ft It is reasonable that prob: 
event, since each resistor is equally likely to 
elementary events similar to that used in Ex 
more rigorous but less readily applied. 


P(A|B) = P(47 Q|5%) is the probability of drawing a 47-1 resistor given 
that the resistor drawn is 5%. P(A|C) = P(47 21100 Q) is the probability 
of drawing a 47-Q resistor given that the resistor drawn is 100 Q; this is 
clearly an impossible event so the probability of it is 0. Finally, 
P(B| C) = P(5%| 100 ©) is the probability of drawing a resistor of 5% toler- 
ance given that the resistor is 100 Q. 

1 


Total Probability . 


The probability :P(A) of any event A defined on a sample space S§ can be 
expressed in terms of conditional probabilities. Suppose we are given N mutually 
exclusive events B,, n= 1, 2,.-6) Ny whose union equals S as illustrated in Figure 


1,4-1. These events satisfy 


B,, 0 B, =D mene, 2,..0,N (1.4-8) 
N 

U 8, =S (1.4-9) 

nat 

We shall prove that 

N 
P(A) = ¥, P(A|B,)P(B,) (1.4-10) 

azt 


which is known as the total probability of event A. 
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Kern 


Figure 1.4-1 Venn diagram of N 
N 
one Si Bn, OB  forallman 


“mutually exclusive events B, and 
another event A. 


Since A m S = A, we may start the proof using (1.4-9) and (1.2-8): 


N N 
Ans=An(Ua)= Yana) (1.4-11) 


a=t ami 


Now the events A m 8B, are mutually exclusive as seen from the Venn diagram 
(Fig. 1.4-1). By applying axiom 3 to these events, we have 


N 
ay= Pan s)= P| Yin By ]= EPA n By (1.4-12) 


nel n=l 


aoe has been used. Finally, (1.4-4) is substituted into (1.4-12) to obtain 


Bayes’ Theoremt 


The definition of conditional probability, as given by (1.4-4), applies to any two 
events. In particular, let B, be one of the events defined above in the subsection 
on total probability. Equation (1.4-4) can be written 


P(B 

P(B,] A) = (14-13) 
if P(A) # 0, or, alternatively, 

P(A | B,) = PEO Se (1.4-14) 


if P(B,) #0. One form of Bayes’ theorem is obtained by equating these two 
expressions: 


PANS ES) 


P(B,| A) = 


(1.4-15) 


¢ The theorem is named for Thomas Bayes (1702-1761), un English philosopher. 
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Another form derives from a substitution of P(A) as given by (1.4-10), 


P(A| B,)P(B, 
PBA) = (A1B)P(B,) 


ee (1.4-16) 
(A| B,)P(B,) + +++ + P(A|By)P(By) 


form = 1,2,...,N. 
An example will serve to illustrate Bayes’ theorem and conditional proba- 
bility. 


OC 


Example 1.4-2 An elementary binary communication system “consists of a 
transmitter that sends one of two possible symbols (a | or a 0) over a channel 
to a receiver. The channel occasionally causes errors to occur so that a | 
shows up at the receiver as a 0, and vice versa. 

The sample space has two elements (0 or 1). We denote by B,, i = 1, 2, 
the events “the symbol before the channel is 1,” and “the symbol before the 
channel is 0,” respectively. Furthermore, define A,, i = 1, 2, as the events “the 
symbol after the channel is 1,” and “the symbol after the channel is 0,” 
respectively. The probabilities that the symbols 1 and 0 are selected for trans- 
mission are assumed to be 


P(B,) = 0.6 and P(B,) bad 0.4 


Conditional probabilities describe the effect the channel has on the (rans- 
mitted symbols. The reception probabilities given a 1 was transmilted are 
assuincd to be 


P(A, | B,) = 09 
P(A,|B,) = 0.1 
The channel is presumed to affect Os in the same manner so 
P(A, | By) = 0.1 
P(A,| By) = 0.9 


In either casc, P(A,{B,) + P(A,|B) = 1 because A, and A, are mutually 
exclusive and are the only “receiver” events (other than the uninteresting 


events @ and S) possible. The channel is often shown diagrammatically as 
illustrated in Figure 1.4-2. Because of its form it is usually called a binary 
symmetric channel. 

From (1.4-10) we obtain the “received” symbol probabilities 


P(A,) = P(A, | B,)P(B,) + P(A, | B2)P(B2) 
= 0.9(0.6) + 0.1(0.4) = 0.58 

P(A,) = P(A2| By) P(B,) + P(A | B2)P(B2) 
= 0.1(0.6) + 0.9(0.4) = 0.42 
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09 


Kho 
P(B,) = 0.6 


P(A N84) 


Figure 1.4-2 Binary symmetric 
P(Ag|B2) communication system — dia- 
grammatical model applicable 

0.9 ; to Example |.4-2. 


P(A) = 04 


From cither (1.4-15) or (1.4-16) we have 
P(A, | B,)P(By) 0.9(0.6) 54 


PW IA=——“py sk 70.58 87?! 
Pb aa) = PAL OD a O O8ST 
Py) As) = PLB) OEY gO 
PB Ay) = ee OO a Og = 0089 


These last two numbers are probabilities of system error while P(B, | A1) 
and P(B,| A,) are probabilities of correct system transmission of symbols. 


In Bayes’ theorem (1.4-16), the probabilities P(B,) are usually referred to as a 
priori probabilities, since they apply to the events B, before the performance of 
the experiment. Similarly, the probabilities P(A | B,) are numbers typically known 
prior to conducting the experiment. Example 1.4-2 described such a case. The 
conditional probabilities are sometimes called transition probabilities in a com- 
munications context. On the other hand, the probabilities P(B,| A) are called a 
posteriori “probabilities, since they apply after the experiment's performance when 
some event 4 is obtained. 


15 INDEPENDENT EVENTS 


In this section we introduce the concept of statistically independent events. 
Although a given problem may involve any number of events in general, it is 
most instructive to consider first the simplest possible case of two events. 


Regie ew 
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Two Events 

Let two events A and B have nonzero probabilities of occurrence; that is, assume 
P(A) # O and P(B) # 0. We call the events statistically independent if the probabil- 
ity of occurrence of one event is not affected by the occurrence of the other event. 
Mathematically, this statement is equivalent to requiring 


P(A| B) = P(A) (1.5-1) 
’ 
for statistically independent events. We also have 
P(B| A) = P(B) (1.5-2) 


for statistically independent events. By substitution of (1.5-1) into (1.4-4), inde- 
pendence also means that the probability of the joint occurrence (intersection) of 
two events must equal the product of the two event probabilities: 


P(A 0 B) = P(A)P(B) (1.5-3) 


Not only is (1.5-3) Cor (1.5-1)] necessary for two events to be independent but it is 
sufficient. As a consequence, (1.5-3) can, and often does, serve as a test of 


independence. 
Statistical independence is fundamental to much of our later work. When 


_ events are independent it will often be found that probability problems are 


greatly simplified. 
It has already been stated that the joint probability of two mutually exclusive 
events is 0: 
P(A 0B) = 0 (1.5-4) 
If the two events have nonzero probabilities of occurrence, then, by comparison 
of (1.5-4) with (1.5-3), we easily establish that two events cannot be both mutually 
exclusive and statistically independent. Hence, in order for two events to be inde- 
pendent they must have an intersection A 1 B# S. 
If a problem involves more than two events, those events satisfying either 
(1.5-3) or (1.5-1) are said to be independent by pairs. 


Fxample 1.5-1 In an experiment, one card is selected from an ordinary 
52-card deck. Define events A as “select a king,” B as “select a jack or 
queen,” and C as “select a heart.” From intuition, these events have probabil- 
ities P(A) = “sa, P(B) = Ys2, and P(C) = "Is : 

It is also easy o state joint probabilities. P(A 7 B) = 0 (it is not possible 
to simultancously select a king and a jack or queen), P(A 9 C) = “egy and 
(BA C) = 452° 


t We shall often use only the word independence to mean statistical independence. 
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We determine whether A, B, and C are independent by pairs by applying 


(1.5-3): 
32 
P(A 1 B) = 0 # P(A)P(B) = ai 
P(ANC)= = = P(A)P(C) =s 
2 
P(BNC)= rr = P(B)P(C) = = 


Thus, A and C are independent as a pair, as are B and C. However, A and B ° 


are not independent, as we might have guessed from the fact that A and B 
are mutually exclusive. ‘ 


eee 


In many practical problems, statistical independence of events is often 
assumed, The justification hinges on there being no apparent physical connection 
between the mechanisms leading to the events. In other cases, probabilities 
assumed for elementary events may lead to independence of other events defined 
from them (Cooper and McGillem, 1971, p. 24). 


Multiple Events 


When more than two events are involved, independence by pairs is not sufficient 
to establish the events as statistically independent, even if every pair satislies 
(1.5-3). : 

In the case of three events’Aj, A,,and Aj, they are said to be independent if, 
and only if, they are independent by all pairs and are also independent as a 
triple; that is, they must satisfy the four equations: 


P(A, 0 A;) = P(A,)P(A;) (1.5-5a) 
P(A, 1 A;) = P(A,)P(A3) (1.5-5b) 
P(A, 0 Aj) = P(A,)P(A3) (1.5-5¢) 
P(A, 1 Az 0 A3) = P(A,)P(A2)P(A3) (1.5-5¢) 


The reader may wonder if satisfaction of (1.5-5d) might be sufficient to guarantee 
independence by puirs, and therefore, satisfaction of all four conditions? The 


answer is no, and supporting examples are relatively easy to construct. The 
reader might try this exercise. 
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More generally, for N events A,, Az,..., Ay to be called statistically inde- 
pendent, we require that all the conditions . 
P(A, A Aj) = P(A)P(A) 
P(A, A Ay OV Ay) = P(A)P(A/) PCAs) 
(1.5-6) 
P(A, 0 A, +++ 0 Ay) = P(A,)P(A2) ++» P(Aw) 


be sulisfied for all 1 <i<j<k<+::<.N. There are 2%-—N—1 of these condi- 
tions (Davenport, 1970, p. 83). 


Example 1,5-2 Consider drawing four cards from an ordinary 52-card deck. 
Let events A,, Az, Ay, Ay define drawing an ace on the first, second, third, 
and fourth cards, respectively. Consider two cases. First, draw the cards 
assuming each is replaced after the draw. Intuition tells us that these events 
are independent so P(A, AN Az A Ay A Ag) = P(A,)P(Az)P(A3)P(Ag) = 
(4/52)* = 3.50(107 5). ; 

On the other hand, suppose we keep each card after it is drawn. We now 
expect these are not independent events. In the general case we may write 


P(A, A Ay AN Ay TO Ay) 
= P(A,)P(A, 9 Ay 9 AglA)) 
= P(A,)P(A2|A,)P(A3 A Agl Ay 2 A)) 
= P(A,)P(A2|A,)P(A3|4, A Az)P(Ag| Ay O Az 9 A;) 


4 3 2 1 
=e: — & 3.69(107° 
52 51 50 49 ( ) 
Thus, we have approximately 9.5-times better chance of drawing four aces 
when cards are replaced than when kept, This is an intuitively satisfying 
result since replacing the ace drawn raises chances for an ace on the suc- 
ceeding draw. 


Properties of Independent Events 


Many properties of independent events may be summarized by the statement: If 
N events A,,A2,..., Ay are independent, then any one of them is independent of 
any event formed by unions, intersections, and complements of the others 
(Papoulis, 1965, p. 42). Several examples of the application of this statement are 
worth listing for illustration. 
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_ For two independent events A, and A; it results that A, is independent of 

A,, A, is independent of A,, and A, is independent of A,. These statements are 
proved as a problem at the end of this chapter. 

For three independent events A,, A,, and A; any one is independent of the 
joint occurrence of the other two. For example 


PLA, 9 (A, 9 A;)) = P(A,)P(A2)P(A3) = P(A,)P(A, 9 A3) (1.5-7) 


with similar slatements possible for the other cases A A (A, 0 A;) and 
Ay (A,  A;). Any one event is also independent of the union of the other 
two, For example 


PLA, A (Az U Ay)) = P(AY)P(Aa As) (1.5-8) 


This result and (1.5-7) do not necessarily hold if the events are only independent 
by pairs. 


«1.66 COMBINED EXPERIMENTS 


All of our work up to this point is related to outcomes from a single experiment. 
Many practical problems arise where such a constrained approach docs not 
apply. One example would be the simultaneous measurement of wind spced and 
barometric pressure at some location and instant in time. Two experiments are 
actually being conducted; one has the outcome “speed”; the other outcome is 
“pressure.” Still another type of problem involves conducting the same experi- 
ment several times, such as flipping a coin N times. In this case there are N per- 
formances of the same experiment. To handle these situations we introduce the 
concept of a combined experiment. 

A combined experiment consists of forming a single experiment by suitably 
combining individual experiments, which we now call subexperiments, Recall that 
an experiment is defined by specifying three quantitics, They are: (1) the applic- 
able sample space, (2) the events defined on the sample space, and (3) the prob- 
abilities of the events. We specify these three quantities below, beginning with the 
sample space, for a combined experiment. 


*Combined Sample Space 


Consider only two subexperiments first. Let S, and S, be the sample spaces of 
the two’ subexperiments and let s, and s, represent the elements of S, and S, 
respectively. We form a new space S, called the combined sample space,t whose 
elements are all the ordered pairs (s,, 52): Thus, if S; has M elements and S, has 
N elements, then S will have MN clements. The combined sample space is 
denoted 


28, 8S; (1.6-1) 


t Also called the cartesian product space in some lexts. 
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Example 1.6-1 If S, corresponds to flipping a coin, then S, = {H, T}, where 
H is the element “heads” and T represents “tails.” Let 5, = {1, 2, 3, 4, 5, 6} 
corresponding {o rolling a single die. The combined sample space S = 
S, x S, becomes 


S = {((H, 1), (H, 2), (H, 3), (A, 4), (1, 5), (H, 6), 
(T, 1), (T, 2), (Ts 3) (7, 4) (Ts 5), (T, 6)} 
In the new space, elements are considered to be single objects, each object 


being a pair of items. 


eS eee ee 


Example 1.6-2 We flip a coin twice, each flip being taken as one sub- 
experiment. The applicable sample spaces are now 


S, = (H, T} 
S,=(H,T} 
=. {(H, H), (H, T), (T, H), (T, T)}: 


2 oe ee 


In this last example, observe that the clement (H, T) is considered different 
from the element (7, H); this fact emphasizes the elements of S are ordered pairs 
of objects. 

The more general situation of N subexperiments is a direct extension of the 
above concepts. For N sample spaces S,, 1" = 1, 2,..., N, having elements s,, the 
combined sample space S is denoted 


S=S,xS,x+1' x Sy (1.6-2) 
and it is the sct of all ordered N-tuples 
(Si, S29 000s Sy) (1.6-3) 


*Events on the Combined Space 


Events may be defined on the combined sample space through their relationship 
with events defined on the subexperiment’ sample spaces. Consider two sub- 
experiments with sample spaces S, and S,. Let A be any event defined on S, and 
B be any event defined on S,, then 


C=AxB (1.6-4) 
is an event defined on S consisting of all pairs (s,, 52)such that ; 
s,EA and s,é€B (1.6-5) 


Since elements of A correspond to clements of the event A x S; defined on S, and 
elements of B correspond to the event S, x Bdefined on S, we easily find that 


Ax B=(A x S83) 0 (S; X B) (1.6-6) 


x 
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Thus, the event defined by the subset of S given by A x B is the intersection of - 


the subsets 4 x S, and S, x B. We consider all subsets of S of the form A x Bas 


events, All intersections and unions of such events are also events (Papoulis, 1965, 
p. 50). 


Example 1.6-3 Let S, = {0 <x < 100} and S, = {0 < y < 50}. The com- 


bined sample space is the set of all pairs of numbers (x, y) withO< x < 100. . 


and 0 < y < 50as illustrated in Figure 1.6-1. For events 
A = {x, <x <x} 
B= {yy <y< ya} : 
where 0 < x, <x; S$ 100 and 0 < y, < y, < 50, the events S, x Band A x 
S, are horizontal and vertical strips as shown. The event 


Ax B= {x, <x <x} x 11 <¥ < ya} 


is the rectangle shown. An event S, x {y = y,} would be a horizontal line. 


In the more general case of N subexperiments with sample spaces S, on 
which events A, are defined, the events on the combined sample space S will all 
be sets of the form 


A, x A,X «tx Ay (1.6-7) 


and unions and intersections of such sets (Papoulis, 1965, pp. 53-54). 


Figure 1.6-1 A combined sample space for two subexperiments. 
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*Probabilities 


To complete the definition of a combined experiment we must assign probabil- 
ilies lo the events defined on the combined sample space S. Consider only two 
subexperiments first, Since all events defined on S will be unions and intersections 
of events of the form A x B, where A cS; and Bc S,, we only need to deter- 
mine P(A x B) for any A and B, We shall only consider the case where 


P(A x B) = P(A)P(B) (1.6-8) 


Subexperiments for which (1.6-8) is valid are called independent experiments. 
To see what elements of S correspond to elements of 4 and B, we only need 
substilute §, for Bor S, for A in (1.6-8): 


P(A x S,) = P(A)P(S,) = P(A) (1.6-9) 
P(S, x B) = P(S,)P(B) = P(B) (1.6-10) 


Thus, elements in the set A x S, correspond to elements of A, and those of 
S, x Bcorrespond to those of B. 
For N independent experiments, the generalization of (1.6-8) becomes 


P(A, x Aq X 18° X Ay) = P(A,)P(Ag) «1° PlAy) (1.6-11) 


where 4, < S,,0 = 1,2,...,N. 

With independent experiments, the above results show that probabilities for 
events defined on S are completely determined from probabilities of events 
defined in the subexperiments. 


1.7 BERNOULLI TRIALS 


We shall close this chapter on probability by considering a very practical 
problem. It involves any experiment for which there are only two possible out- 
comes on any trial. Examples of such an experiment are numerous: flipping a 
coin, hitting or missing the target in artillery, passing or failing an exam, re- 
ceiving a 0 or a | in a computer bit stream, or winning or losing in a game of 
chance, are just a few, 

For this type of experiment, we let A be the elementary event having one of 
the two possible outcomes as its element. A is the only other possible elementary 
event. Specifically, we shall repeat the basic experiment N times and determine 
the probability that A is observed exactly k times out of the N trials. Such re- 
peated experiments are called Bernoulli trials.t Those readers familiar with com- 
bined experiments will recognize this experiment ‘as the combination of N 
identical subexperiments. For readers who omitted the section on combined 
experiments, we shall develop the problem so that the omission will not impair 
their understanding of the material, 


+ After the Swiss mathematician Jacob Bernoulli (1654-1705). 
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Assume that elementary events are statistically independent for every trial. 
Let event A occur on any given trial with probability 


P(A) = p (1.7-1) 
The event A then has probability 
P(A) =l—p (1.7-2) 


After N trials of the basic experiment, one particular sequence of oulcomes has A 
occurring k times, followed by A occurring N —k times.f Because of assumed 
statistical independence of trials, the probability of this one sequence is 


P(A)P(A) © + P(A) P(A)PCA) «++ P(A) = PAL — pe (1.7-3) 
TSO eee 


k terms N —k terms 


Now there are clearly other particular sequences that will yicld k events A 
and N—k events A.f The probability of cach of these sequences is given by 
(1.7-3). Since (he sum of all such probabilitics will be the desired probability of A 
occurring exactly k times in N trials, we only need find the number of such 
sequences. Some thought will reveal that this is the number of ways of taking k 
objects at a time from N objects. From combinatorial analysis, the number is 


known to be 
N N! 
eres (1.7-4) 


The quantity (f) is called the binomial coefficient. It is sometimes given the symbol 
ch. 
From the product of (1.7-4) and (1.7-3) we finally obtain 


N 
P{A occurs exactly k limes} = (“re — py-* (1.7-5) 


Example 1.7-1 A submarine attempts to sink an aircraft carrier. It will be 
successful only if two or more torpedocs hit the carrier. If the sub fires three 
torpedoes and the probability of a hit is 0.4 for each torpedo, what is the 
probability that the carrier will be sunk? 


+ This particular sequence corresponds 10 one N-dimensional element in the combined sample 


space S. , , 
} All such sequences define all the elements of S that satisfy the event {A occurs exacily k times in 


N trials} defined on the combined sample space. 
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Define the event A = {torpedo hits}. Then P(A) = 0.4, and N = 3. Prob- 
avilitics are found from (1.7-5): 


P{exactly no hits} = (your — 0.4)? = 0,216 


0 


3 


P{exactly one hit} = 


Joa) — 0.4)? = 0.432 
(0.4)?(1 — 0.4)! = 0.288 


P{exactly three hits} = (0.4)°(1 — 0.4)° = 0.064 


P{exactly two hits} = ( 


The answer we desire is 


P{carrier sunk} = P{two or more hits} 
= P{exactly two hits} + P{cxactly three hits} 
= 0,352 


a 
Example 1.7-2 In a culture used for biological research the growth of un- 
avoidable bacteria occasionally spoils results of an experiment that requires 
at Ieast three out of four cultures to be unspoiled to obtain a single datum 
point. Experience has shown that about 6 of every 100 cultures are randomly 
spoiled by the bateria. If the experiment requires three simultaneously 
derived, unspoiled data points for success, we find the probability of success 
for any given set of 12 cultures (three data points of four cultures each). 

We treat individual datum points first as a Bernoulli trial problem with 
N = 4and p = P{good culture} = *“/,9 = 0.94. Here 


P{valid datum point} = P{3 good cultures} + P{4 good cultures} 
4 4 
= (S}oss — 0.94)! + (‘joss — 0.94)° = 0.98 


Finally, we treat the required three data points as a Bernoulli trial 
problem with N = 3 and p = P{valid datum point} = 0.98. Now 


P{successful experiment} = P{3 valid data points} 


= (joss — 0.98)° = 0.941. 


Thus, the given experiment will be successful about 94.1 percent of the time. 


eae oe eS eee 


x Met 


| 


gars Ft 
ae it 


Tn aT 
xt nae Sl sea eee a 
SF i. 


28 PROBABILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 
PROBLEMS 
1-1 Specify the following sets by the rule method, 


A = {1, 2, 3}, B= {8 10, 12, 14}, C= {1, 3, 5, 7,...} 


1-2 Use the tabular method to specify a class of sets for the sets of Problem 1-1, 
1-3 State whether the following sets are countable or uncountable, or, finite or 


infinite. A = {1}, B = {x = 1}, C = {0 < integers}, D = {children in public school *, 
No. 5}, E = {girls in public school No. 5}, F = {girls in class in public school ; 


No. 5 at 3:00 a.m}, G= {all lengths not exceeding one meter}, H = 
{-25< xs —3},] = {-2,-l,1 <x <2}. 

1-4 For each set of Problem 1-3, determine if it is equal t6, or a subset of, any of ° 
the other sets. i 

1-5 State every possible subset of the set of letters {a, b, c, dj. 


1-6 A thermometer measures temperatures from —40 to 130°F (—40 to 54.4°C). 
(a) State a universal set to describe temperature measurements. Specify ° 
subsets for: 
(b) Temperature measurements not exceeding water’s freezing point, and 


(c) Measurements exceeding the freezing point but not exceeding 100°F 
(37.8°C). 


*1.7 Prove that a set with N elements has 2" subsets. 


1-8 A random noise voltage at a given time may have any value from —10 to 
10 V. 


(a) What is the universal! set describing noise voltage? ; 

(b) Find a set to describe the voltages available from a half-wave rectifier for ° 
positive voltages that has a linear output-input voltage characteristic. 

(c) Repeat parts (a) and (5) if a de voltage of —3 V is'added to the random 
noise. 
1-9 Show thatC cAifC co Band Bc A, 
1-10 Two sets are given by A = {—6, —4, —0.5, 0, 1.6, 8} and B = {—0.5, 0, 1, 2, 
4). Find: 

(a) A-B (b|)}B-A (c)AUB (Wd) ANB 
1-11 A univeral set is given as S = {2, 4, 6, 8, 10, 12}. Define two subsets as 
A = {2, 4, 10} and B = {4, 6, 8, 10}. Determine the following: 

(a) A=S—A (b) A-— Band B-A (c) AUB (d) ANB 

(ec) ANB 
1-12 Using Venn diagrams for three sets A, B, and C, shade the areas corre- 
sponding to the sets: ae 

(a) (AUB)-—C (bt) BNA (DANBAC W)(AUB)AC 7 
1-13 Sketch a Venn diagram for three events where AN B# OG, BNC#@, 
CnNAF GD, butANBACH=Z. 
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1-14 Use Venn diagrams to show that the following identities are true: 
(a) (AU B)AC=C-LANCU(BAC) _ 
(b) (AUBUC)-(ANBnC)= (An B)U (BAC) U(C a A) 
() (AA BNC)=AUBUC 
1-15 Use Venn diagrams to prove De Morgan's laws (A UB) = An B and 
(AA B)=A VB. 
a 16 A universal set is S={-20<s< —4}, If A={-10<5 —5} and 


=(-71<s< —4}, find: 

(a) AUB 

(b) ANB 

(c) A third set C such that the sets A a Cand Bn C are as large as possible 
while the smallest element in C is —9. 


(d) What is the set An Bn C? 


Ga) Use De Morgan's laws to show that: 
(a) AA(BUC)S=(AV B)n(AvVC) 
(bs) (AN BAC)=AVBVC 


In each case check your results using a Venn diagram. 


1-18 A die is tossed. Find the probabilities of the events A = {odd number shows 

up}, B = {number larger than 3 shows up}, A U B, and An B, 

1-19 An a game of dice, a “ shooter” can win outright if the sum of the two 
mbers showing up is either 7 or 11 when two dice are thrown, What is his 

prebability of winning outright? 


Cas A pointer is spun on a fair wheel of chance having its periphery labeled 
Fom 0 to 100. 
(a) What is the sample space for (his experiment? 
(b) What is the probability that the pointer will stop between 20 and 35? 
(c) What is the probability that the wheel will stop on 58? 
(129) An experiment has a sample space with 10 equally likely elements S = {a,, 
/..., Ayo}. Three events are defined as A = (ay, a5, ao}, B= {ay, 42, 46, M9}, 
and C = {dg, dg}. Find the probabilities of: 
(a) AUC 
(bs) BUC 
(ec) AN(BUC) 
(dd) AUB 
(ec) (AUB)AC 


1-22 Let A be an arbitrary event. Show that P(A) = 1 — P(A). 
1-23 An experiment consists of rolling a single die. Two events are defined as: 
A = {a 6 shows up} and B = {a 2 ora 5 shows up}. 
(a) Find P(A) and P(B). 
~{b) Define a third event C so that P(C) = 1 — P(A) — P(B). 
Cash a box there are 500 colored balls: 75 black, 150 green, 175 red, 70 white, 
3 


0 blue. What are the probabilities of sclecting a ball of each color? 
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a. 
f 


(1-33 Rework Example 1.4-2 if P(B,) = 0.7, P(B;) = 0.3, P(A, | By) = P(A;] Ba) = 
0 and P(A,| B,) = P(A, | B,) = 9. What type of channel does this system have? 
(1-34.A company sells high fidelity amplifiers capable of generating 10, 25, and 
‘W of audio power. It has on hand 100 of the 10-W units, of which 15% are 
defective, 70 of the 25-W units with 10% defective, and 30 of the 50-W units with 
10% defective. 
(a) What is the probability that an amplifier sold from the 10-W units is 
defective? : 
(b) If each wattage amplifier sells with equal likelihood, what is the probabil- 
ity of a randomly selected unit being 50 W and defective? 
(c) What is the probability that a unit randomly sclected for sale is defective? 
(139) missile can be accidentally launched if two relays A and B both have 
Hed. The probabilities of A and B failing are known to be 0.01 and 0,03 respec- 
tively. It is also known that B is more likely to fail (probability 0.06) if A has 
failed. 
(a) What is the probability of an accidental missile launch? 
(by What is the probability that A will fail if B has failed? 
(c) Are the events “A fails” and “ B fails” statistically independent? 


1-36 Determine whether the three events A, B, and C of Example 1.4-1 are sta- 
tistically independent. 
1-37 List the various equations that four events A, Aq, Ay, and A, must satisfy 


j are to be statistically independent. 
= (38 2 iven that two events A, and A, are statistically independent, show that: 
rs (a) A, is independent of A, ane ; : 
(b) A, is independent of A, 
(c) A, is independent of A, 
*1.39 An experiment consists of randomly selecting one of five cities on Florida’s 
west coast for a vacation. Another experiment consists of selecting at random one 
of four acceptable motels in which to stay. Define sample spaces S, and S, for the 


1-25 A single card is drawn from a 52-card deck. 
(a) What is the probability that the card is a jack? ne 
(b) What is the probability the card will be a 5 or smaller? ; 
(c) What is the probability that the card is a red 10? 


1-26) Two cards are drawn from a 52-card deck (the first is not replaced). 

(a) Given the first card is a queen, what is the probability that the second is 
also a queen? , 

(b) Repeat part (a) for the first card a queen and the second carda7. 
_-(c) What is the probability that both cards will be a queen? 


1-27/An ordinary 52-card deck is thoroughly shuffled. You are dealt four cards 
lip. What is the probability that all four cards are sevens? 


1-28 For the resistor selection experiment of Example 1.4-1, define event D as 
“draw i 22-0 resistor,” and £ as “draw a resistor with 10% tolerance.” Find 


P(D), P(E), P(D a E), P(D| E), and P(E] D). 


1-29 For the resistor selection experiment of Example 1.4-1, define two mutually 
exclusive events B, and B, such that By U B,= 5S. 
(a) Use the total probability theorem to find the probability of the event 


“ sclect a 22-M resistor,” denoted D. 
(b) Use Bayes’ theorem to find the probability that the resistor sclected had 


5% tolerance, given it was 22 2. 


(1-30) In three boxes there are capacitors as shown in Table P 1-30, An experiment 
Consists of first randomly selecting a box, assuming each has the same likelihood 
of selection, and then selecting a capacitor from the chosen box. 

(a) What is the probability of selecting a 0.01-F capacitor, given that box 2 
is selected? 

(b) Ifa 0.01-nF capacitor is selected, what is the probability it came from box 
32 (Hint; Use Bayes’ and total probability theorems.) 


Table P1-30 Capacitors . : 
having the two subexperiments. 


*1.40 Sketch the area in the combined sample space of Example 1.6-3 correspond- 
ing to the event A x: B where: 


Number in box 


a 


SFR OTIS VES EE ES pL 


Value (iF) { 2 3 Totals 
Nalue WP) : 
x (a) A= {l0<x's 15} and B = {20<y < 50} 
0.01 20 95 25 140 
0.1 a ae a ee (b) A = {x = 40} and B = (5 <y S40} 
1.0 70 go «145295 i ) A production line manufactures 5-gal (18.93-liter) gasoline cans to a volume 
Totals 145 210-—«24S-—«600 folerance of 5%. The probability of any one can being out of tolerance is 0.03. If 
— ‘ four cans are selected at random: 
a eh ’ 3 (a) What is the probability they are all out of tolerance? 
1-31 For Problem 1-30, list the nine conditional probabilitics of capacitor sclec- a (b) What is the probability of exactly (wo being out? 
Sion, given certain box selections. if (c) What is the probability that all are in tolerance? 
. S: c a 
1-32} Rework Example 1.4-2 if P(B,) = 0.6, P(B2) = 0.4, P(A, | Bi) = P(A, | B,) = é 1-42 Spacecraft are expected to land in a prescribed recovery zone 80% of the 


i 
i 
| 


055, and P(A, | By) = P(A, |B) = 0.05. time. Over a period of time, six spacecraft land. 


two experiments and a combined space S = S, x S; for the combined experiment : 
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(a) Find the probability that none lands in the prescribed zone. 

(6) Find the probability that at least one will land in the prescribed zone. 

(c) The landing program‘is‘called successful if the probability is 0.9 that three 
or more out of six spacecraft will land in the prescribed zone. Is (he program suc- 
cessful? 

1-43 In the submarine problem of Example 1.7-1, find the probabilities of sinking 
the carrier when fewer (N = 2) or more (N = 4) torpedoes are fired. 


ADDITIONAL PROBLEMS 


1-44 Use the tabular method to define a set A that contains all integers with 
magnitudes not exceeding 7. Define a second set B having odd integers larger 
than —2 and not larger than 5. Determine if A ¢ Band if Be A. 

as A set A has three elements a,, a,, and a,. Determine all possible subsets 
Oca: : 

1-46 Shade Venn diagrams to illustrate each of the following sets: (a) (A U B) a 
CK) (AN BLE ()(AU BUCA D),M(ANBAC)UBACA D). 
1-47 A universal set S is comprised of all points in a rectangular area defined by 
Osx<s3 and Os ys4. Define three sets by 4 = {y < 3(x —1)/2}, B= 


{y2 1}, and C = {y23—-x}. Shade in Venn diagrams corresponding to the . 


sets (a) A VW BOC,and(b)C ON BO A. 
1-48 The take-off-roll distance for aircraft at a certain airport can be any number 
from 80 m to 1750 m. Propeller aircraft require from 80 m to 1050 m while jets 
use from 950 m to 1750 m, The overall runway is 2000 m. 

(a) Determine sets A, B, and C defined as “propeller aircraft take-off dis- 


tances,” “jet aircraft take-off distances,” and “runway length safety margin,” 
respectively. 


(b) Determine the set A 4 B and give its physical significance, 

(c) What is the meaning of the set-A U B? 

(d) What are the meanings of the sets A U B U.Cand A vu B? ; 
1-49 Prove that DeMorgan’s law (1.2-13) can be extended to N events A,,i=1, 
2,..., N as follows . 

(4, NA, O° A Ay) = (A, UA, UU Ay). 
1-50 Work Problem 1-49 for (1.2-12) to prove 
(A, UV A, Ut U Ay) = (A, 0 AZ NO Ay), 
1-51 A pair of fair dice are thrown in a gambling problem. Person A wins if the 


sum of numbers showing up is six or less and one of the dice shows four, Person - 


B wins if the sum is five or more and one of the dice shows a four, Find: (a) The 


probability that A wins, (b) the probability of B winning, and (c) the probability 


that both A and B win. 


1-52 You (person A) and two others (B and C) each toss a fair coin in a two-step ‘ 
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gambling game. In step | the person whose toss is not a match to either of the 
other two is “odd man out.” Only the remaining two whose coins match go on 
to step 2 to resolve the ultimate winner. 

(a) What is the probability you will advance to step 2 after the first toss? 

(b) What is the probability you will be out after the first toss? 

(c) What is the probability that no one will be out after the first toss? 

*1.53 The communication system of Example 1.4-2 is to be extended to the case 
of three transmitted symbols 0, 1, and 2. Define appropriate events A, and B;, 
i= 1, 2, 3, to represent symbols after and before the channel, respectively. 
Assume channel transition probabilities are all equal at P(A,| B,) = @.1, i #j, and 
are P(A,| B,) = 0.8 for i =j = 1, 2, 3, while symbol transmission probabilities are 
P(B,) = 0.5, P(B,) = 0.3, and P(B3) = 0.2. 

(a) Sketch the diagram analogous to Fig. 1.4-2. 

(b) Compute received symbol probabilities P(A,), P(A2), and P(A). 

(c) Compute the a posteriori probabilities for this system. 

(d) Repeat parts (b) and (c) for all transmission symbol probabilities equal. 
Note the effect. 
1-54 Show that there are 2% — N — | equations required in (1.5-6). (int: Recall 
that the binomial coefficient is the number of combinations of N things taken n 
at a time.) 
1-55 A student is known to arrive late for a class 40% of the time. If the class 
meets five times each weck find: (a) the probability the student is late for at least 
three classes in a given week, and (b) the probability the student will not be late 
at all during a given week. 
1-56 An airline in a small city has five departures each day. It is known that any 
given flight has a probability of 0.3 of departing late. For any given day find the 
probabilities that: (a) no flights depart late, (b) all flights depart late, and (c) three 
or more depart on time. 
1-57 The local manager of the airline of Problem 1-56 desires to make sure that 
90% of flights leave on time, What is the largest probabilily of being late that the 
individual flights can have if the goal is to be achieved? Will the operation have 
to be improved significantly ? 
1-58 A man wins in a gambling game ‘f he gets (wo heads in five Nips of a biased 
coin. The probability of getting a head with the coin is 0.7. 

(a) Find the probability the man will win. Should he play this game? 

(b) What is his probability of winning if he wins by getting at least four 
heads in five flips? Should he play this new game? 

*1-59 A rifleman can achieve a “marksman” award if he passes a test. He is 
allowed to fire six shots at a target’s bull’s eye. If he hits:the bull’s eye with at 
least five of his six shots he wins a set. He becomes a marksman only if he can 
repeat the feat three times straight, that is, if he can win three straight sets. If his 
probability is 0.8 of hitting a bull’s eye on any one shot, find the probabilities of 
his: (a) winning a set, and (b) becoming a marksman. 
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THE RANDOM VARIABLE 


2.0 INTRODUCTION 


In the previous chapter we introduced the concept of an event to describe charac- 
teristics of outcomes of an experiment. Events allowed us more flexibility in 
determining properties of an experiment than could be obtained by considering 
only the outcomes themselves. An event could be almost anything from 
“ descriptive,” such as “draw a spade,” to numerical, such as “ the outcome is 3.” 

In this chapter, we introduce a new concept that will allow events to be 
defined in a more consistent manner, they will always be numerical, The new 
concept is that of a random variable, and it will constitute a powerful tool in the 
solution of practical probabilistic problems. 


2.1 THE RANDOM VARIABLE CONCEPT 


Definition of a Random Variable 


We define a real random variablet as a real function of the elements of a sample 
space S, We shall represent a random variable by a capital letter (such as W, X, 
or Y) and any particular valuc of the random variable by a lowercase letter (such 


+ Complex random variables are considered in Chapter 5. 


M 
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as w, x, or y). Thus, given an experiment defined by a sample space S with ele- 
ments s, we assign to every $a real number 


X{s) (2.1-1) 


according to some rule and call X(s) a random variable. 


A random variable X can be considered to be a function that maps all ele- 


ments of the sample space into points on the real line or some parts thereof. We 
illustrate, by two examples, the mapping ofa random variable. 


dxample 2.1-1 An experiment consists of rolling a dic and flipping 4 coin, 
The applicable sample space is illustrated in Figure 2.1-1. Let the random 
variable be a function X chosen such that (1) a coin head (H) outcome corre- 
sponds to positive values of X that are equal to the numbers that show up on 
the dix, and (2) a coin tail (T) outcome corresponds to negative values of X 
that are equal in magnitude to twice the number that shows on the die. Here 
X maps the sample space of 12 elements into 12 values of X from —12 to 6 
ns shown In Figure 2.1-1, 


Example 2.1-2 Figure 2,1-2 illustrates an experiment where the pointer on a 
whec! of chance is spun. The possible outcomes are the numbers from 0 to 12 
marked on the wheel. The sample space consists of the numbers in the set 
{0 <s < 12}. We define a random variable by the function 


X=X(s)=s? 


Points in S now map onto the real line as the set {0 < x < 144}. 


Figure 2.1-1 A random variable map- 
ping of a sample space. 
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12 


09 36 100 144 200 x 


to Example 2.1-2. 


As seen in these two examples, a random variable is a function that maps - 


each point in S into some point on the real line. It is not necessary that the 
sample-space points map uniquely, however. More than one point in S may map 


into a single value of X, For example, in the extreme case, we might map all six * 


points in the sample space for the experiment “throw a die and observe the 
number that shows up” into the one point X = 2. , 


Conditions for a Function to be a Random Variable 


Thus, a random variable may be almost any function we wish. We shall, however, 


require that it not be multivalued. That is, every point in S must correspond to ° - 


only one value of the random variable. 

Moreover, we shall require that two additional conditions be satisfied in 
order that a function X be a random variable (Papoulis, 1965, p. 88). First, the 
set {X < x} shall be an event for any real number x. The satisfaction of this con- 
dition will be no trouble in practical problems. This set corresponds to those 
points s in the sample space for which the random variable X(s) does not excced 
the number x, The probability of this event, denoted by P{X < x}, is equal to the 
sum of the probabilities of all the clementary events corresponding to {X < x}. 

The second condition we require is that the probabilities of the events 
{X = co} and {X = —0o} be 0: 

P{(X=-ao}=0 P{X =a} =0 
This condition does not prevent X from being either —oo or oo for some values 
of s; it only requires that the probability of the set of those s be zero. 


Figure 2.1-2 Mapping applicable 


(2.1-2) s | 
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Discrete and Continuous Random Variables 


A discrete random variable is one having only discrete valucs. Example 2.1-! illus- 
trated a discrete random variable. The sample space for a discrete random vari- 
able can be discrete, continuous, or even a mixture of discrete and continuous 
points. For example, the “wheel of chance” of Example 2.1-2 has a continuous 
sample space, but we could define a discrete random variable as having the value 
1 for the set of outcomes {0 <s < 6} and —1! for {6 <s < 12}. The result isa 
discrete random variable defined on a continuous sample space. 

A continuous random variable is one having a continuous range of values. It 
cannot be produced from a discrete sample space because of our requirement 
that all-random variables be single-valued functions of all sample-space points. 
Similarly, a purely continuous random variable cannot result from a mixed 
sample space because of the presence of the discrete portion of the sample spice. 
The random variable of Example 2.1-2 is continuous. 


Mixed Random Variable 


A mixed random variable is one for which some of its values are discrete and some 
are continuous. The mixed case is usually the least important type of random 
variable, but it occurs in some problems of practical significance. 


2.2 DISTRIBUTION FUNCTION 


The probability P{X < x} is the probability of the event {X < x}. It is a number 
that depends on x; that is, it is a function of x. We call this function, denoted 
F(x), the cumulative probability distribution function of the random variable x, 
Thus, 


F,(x) = P(X < x} (2.2-1) 


We shall often call F(x) just the distribution function of X, The argument x is any 
real number ranging from — © to oo. 

The distribution function has some specific properties derived from the fact 
that Fy(x) is a probability. These are: 


(1) Fx(—co) = 0 (2.2-2a) 
(2) Fx(co) = 1 (2.2-2b 
(3) O< Fy(x) <1 (2.2-2c) 
(4) Fy(x,)S Fy.) if x, < xy (2.2-2c) 
(5) P{xy <X < x2} = Fy(x2) — Fx(x}) (2.2-2¢) 
(6) Fy(x*) = Fy(x) (2.2-2f) 


t We use the notation x* to imply x + ¢ where ¢ > 0 is infinitesimally small; that is, e—» 0, 


note 
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The first three of these properties are casy to justify, and the reader should justify 


them as an exercise. The fourth states that Fy(x) is a nondecreasing function of x. *** p 


The fifth property states that the probability that X will have values larger than 
some number x, but not exceeding another number x, is equal to the difference 
in Fy(x) evaluated at the two points. Il is justified from the fact that the events 
{X sx,} and {x, < X <x,} are mutually exclusive, so the probability of the 
event {X sx} ={XS x) U(x, <XS x,} is the sum of the probabilities 
P{X < x,} and Pix, < XS X}. The sixth property states that F(x) is a func- 
tion continuous from the right. 

Propertics 1, 2, 4, and 6 may be used as tests to determine if some function, 
say Gx(x), could be a valid distribution function. If so, all four tests must be 
passed. 

if X is a discrete random variable, consideration of its distribution function 
defined by (2.2-1) shows that Fy(x) must have a stairstep form, such as shown in 
Figure 2.2-la. The amplitude of a step will equal the probability of occurrence of 
the value of XY where the step occurs. If the values of X arc denoted x,, we may 
write Fy(x) as 


N 
F(x) = ¥ P(X = x}ulx — x) (2.2-3) 
ist 
where (+) is the unit-step function defined byt 
1 x20 
u(x) = {j ee (2.2-4) 


and N may be infinite for some random variables. By introducing the shortened 
notation 
P(x) = P{X = xi} (2.2-5) 


(2.2-3) can be written as 


N 
F(x) = Y, Plxdulx — x) (2.2-6) 
i= 


We next consider an example that illustrates the distribution function of a 
discrete random variable. 


en Ea a lan 
Example 2.2-1 Let X have the discrete values in the sect {—!, —0.5, 0.7, 1.5, 
3}. The corresponding probabilities are assumed to be {0.1, 0.2, 0.1, 0.4, 0.2}. 
Now P(X < —1} = 0 because there are no sample space points in the set 
{X < —1}. Only when X = —1 do we obtain one outcome. Thus, there is an 
immediate jump in probability of 0.1 in the function F(x) al the point 
yea —l. For -Il<<< —0.5, there are no additional sample space points so 
F (x) remains constant at the valuc 0.1. Atx = —0.5 there is another jump of 


t This definition differs slightly from (A-5) by including the equality so that idx) satisfies (2.2-2/). 
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x 
(b) 
Figure 2.2-1 Distribution function (a) and density function ()) applicable to the dscrete random vari- 


able of Example 2.2-1. {Adapted from Peebles (1976) with permission of publishers Addison-Wesley, 
Advanced Book Program.} 


0.2 in F(x). This process continues until all points are included. F(x) then 
equals 1.0 for all x above the last point. Figure 2.2-1a illustrates Fy(x) for this 
discrete random variable. 


A continuous random variable will have a continuous distribution function. 
We consider an example for which Fy(x) is the continuous function shown in 
Figure 2.2-2a. 


Example 2.2-2 We return to the fair wheel-of-chance experiment. Let the 
wheel be numbered from 0 to 12 as shown in Figure 2.1-2. Clearly the prob- 
ability of the event {X < 0} is 0 because there are no sample space points in 
this set. For 0 < x < 12 the probability of {0 <X < x} will increase linearly 
with x fora fair wheel. Thus, F(x) will behave as shown in Figure 2.2-2a. 


The distribution function of a mixed random variable will be a sum of two 
parts, one of stairstep form, the other continuous. : 
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Fy(x) 
1.0 
| 
0.5 | 
{ 
! ay 
| ; 
| 
0 6 2° x 


(a) 


Figure 2.2-2 Distribution function (a) and 

density function (b) applicable to the continuous 

tandom variable of Example 2.2-2, [Adapted 

0 6 12 x from Peebles (1976) with permission of publishers 
(b) Addison-Wesley, Advanced Book Program.) 


2.3 DENSITY FUNCTION ; 


The probability density finetion, denoted by fy(x), is defined as the derivative of 
the distribution function: 


filo) = Ed 


he (2.3-1) 


We often call f,(x) just the density function of the random variable X. 


Existence 


If the derivative of Fy(x) exists then ,(x) exists and is given by (2.3-1). There may, 
however, be places where dF,(x)/dx is not defined. For example, a continyous 
random variable will have a continuous distribution F,(x), but Fy(x) may have 
corners (points of abrupt change in slope). The distribution shown in Figure 
2,2-2a is such a function. For such cases, we plot f;(x) as a function with step- 
type discontinuities (such as in Figure 2.2-2b). We shall assume that the number 
of points where F(x) is not differentiable is countable. 
For discrete random variables having a stairstep form of distribution func- 
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tion, we introduce the concept of the unit-impulse function d(x) to describe the 
derivative of Fy(x) at its stairstep points. The unit-impulse function and ils 
properties are reviewed in Appendix A, It is shown there that d(x) may be defined 
by its integral property 


(Xo) = \ (x)6(x — Xo) dx (2.3-2) 


where $(x) is any function continuous at the point x = Xo; 5(x) can beinterpreted 
as a “function” with infinite amplitude, area of unity, and zero duration, The 
unit-impulse and the unit-step functions are related by 


8(x) = ay (2.3-3) 
or 
i (2) dé = u(x) (2.3-4) 
| 


The more general impulse function is shown symbolically as a vertical arrow 
occurring at the point x = xy and having an amplitude equal to the amplitude of 
the step function for which it is the derivative, 

We return to the case of a discrete random variable and differentiate Fy(x), 
as given by (2.2-6), lo obtain 


N 
Sx(x) = », P(X )d(x — x1) (2.3-5) 


tat 


Thus, the density function for a discrete random variable exists in the sense that 
we use impulse functions to describe the derivative of F(x) at its stairstep points, 
Figure 2.2-1b is an example of the density function for the random variable 
having the function of Figure 2.2-1a as its distribution. 

A physical interpretation of (2.3-5) is readily achieved. Clearly, the probabil- 
ity of X having one of its particular values, say x,, is a number P(x,). If this prob- 
ability is assigned to the point x,, then the density of probability is infinite 
because i point has no “width” on the x axis, The infinite “amplitude” of the 
impulse function describes this infinite density. The “size” of the density of prob- 
ability at x = x, is accounted for by the scale factor P(x,) giving P(x))5(x — x;) for 
the density at the point x = x). 
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require that the density function be nonnegative and have an area of unity. These 
(wo properties may also be used as tests to see if some function, say gx(x), can be 


ta 


Properties of Density Functions ia 
Several properties that f(x) satisfics may be stated: « 
(1) OS f,(x) all x (2.3-6a)." ; 
(2) [" fel) dx = | (2.3-6b) ; | 
(3) Fy) = [7 40 dé (2.3-60) diaimaee | | 
(4) P(x, <X sx}= [ Sxl) dx (2.3-6d) 10 : | ! : 
Proofs of these properties are left to the meee exercises. Propertics 1 and 2 a | 
| | 
Je 2a 


Ri 


Next, by using (2.3-6c), we obtain Thus, the event {45<X s 6.7} has a probability of 0.2288 or 22.88%. 


0 Xg-a>Xx 


fy. 
a valid probability density function. Both tests must be satisfied for validity. Ht - Figure 23-1 A. possible probability | 
Property 3 is just another way of writing (2.3-1) and serves as (he link belween i ak pe ene Ps Sen preva " ang. ee 4 
F(x) and f,(x). Properly 4 relates the probability that X will have values from x, ay o function (b) applicable to Exaunple 2.31 \ 
to, and including, x, to the density function. ay 
ie Example 2.3-2 Suppose a random variable is known to have the triangular of 
Example 2.3-1 Let us test the function gx(x) shown in Figure 23-tato sce if Be Pee, conte ee ira example with xo = 8, a= 5 and a BR 
it can be a valid density function. It obviously satisfies property | since itis fae a se 
nonnegative. Its area is ax which must equal unity to satisfy property 2. ° i: 0 3>x213 ne 
Therefore a = t/a is necessary if g(x) is to be a density. “7 Sxhx) = 4 (% = 3)/25 3x <8 ak 
Suppose a = I/a. To find the applicable distribution function we first RR 0.2 —(x - 8/25. 8sx<13 tes 
write ; pee A We shall use this probability density in (2.3-6d) to find the probability that X 
0 Xp ~&>XSXo +o . has values greater than 4.5 but not greater than 6.7. The probability is 
1 tot 6.7 
gx) = ai (t= Xo + 4) Xp — ASX <Xo a pas<xs6n=[ [oe — 3)/25) dx 
wv tg 4,5 { 
1 ot ig F 1 fx? 6.7 
Par Xg SX <Xo tea ye = 75 yee - = 0.2288 
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A random variable X is called gaussiant if its density function has the form 


1 
Sx(x) = Sanat 


¢ After the German mathematician Johann Friedrich Carl Gauss (1777-1855). The gaussian 
density is often called the normal density. 


GAN) =4 0 * 1 1 | 2 
oe gxl€) db = 5 + (¥ — Xo) — Fya & — 0) m(x-ax)/2ext 
xo Xo SX < Xo +a . , : on 


1 Xgtasx 


This function is plotted in Figure 2.3-1b. 
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ty (x) 


Figure 2.4-1 Density (a) and 
distribution (4) functions of ww 
(b) gaussian random variable, 


Q ay - Oy ay ay toy x 


where oy >Qand ~ 0 < ay < 0 are real constants. This function is sketched in 
Figure 2.4-la, Its maximum value (2m0%)~'/? occurs at x = ay. Its “spread” 
about the point x = a, is related to ay. The function decreases to 0.607 times its 
maximum at x = ay + oy and x = ay — oy, 

The gaussian density is the most important of all densities. It enters into 
nearly all areas of engineering and science. We shall encounter the gaussian 
random variable frequently in later work when we discuss some important types 
of noise. 

The distribution function is found from (2.3-6c) using (2.4-1). The integral is 


F(x) = 


This integral has no known closed-form solution and must be evaluated by 
numerical methods. To make the results generally available, we could develop a 
set of tables of F(x) for various x with ay and oy as parameters. However, this 
approach has limited value because there is an infinite number of possible com- 


l Po Ee a 
Al ikea (2.4-2) 
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binations of ay and ay, which requires an infinite number of tables, A better 
approach is possible where only one table of Fy(x) is developed Chat corresponds 
lo normalized (specific) values of ay and oy. We then show that the one table can 
be used in the general case where ay and oy can be arbitrary. 

We start by first selecting the normalized case where ay = 0 and oy = 1. 
Denote the corresponding distribution function by F(x). From (2.4-2), F(x) is 


1 x 
F(x) = —= y 82 2.4-3) 
(x) a ie de ( 


. 


which is a function of x only. This function is tabularized in Appendix B for . 
x > 0. For negative values of x we use the relationship 


F(—x) = 1 ~ Fix) (24-4) 


To show that the general distribution function F(x) of (2.4-2) can be found 
in terms of F(x) of (2.4-3), we make the variable change 


n= (F — ayVox (2.4-5) 
in (2.4-2) to obtain 
{ {x ~aylax nn ) an 6) 
yx) = = | err du (2.4-0 
J 20 J-w 


From (2.4-3), this expression is clearly equivalent to 


Fy(x) = (2) (2.4-7) 
a 


x 


Figure 2.4-1b depicts the behavior of F(x). 
We consider two examples to illustrate the application of (2.4-7). 


i 


Example 2.4-1 We find the probability of the event {X < 5.5} for a gaussian 


random variable having ay = 3and oy = 2. 
Here (x — ax)/oy = (5.5-3)/2 = 1.25, From (2.4-7) and the definition of 
Fy(x) 


P{X < 5.5} = F,(5.5) = F(1.25) 
By using the table in Appendix B 
P{X < 5.5} = F(1.25) = 0.8944 
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Example 2.4-2 Assume that the height of clouds above the ground at some 
location is a gaussian random variable X with ay = 1830 m and gy = 460 m. 
We find the probability that clouds will be higher than 2750 m (about 
9000 ft). From (2.4-7) and Appendix B: 
P{X > 2750} = 1—- P{X < 2750} =1—- F ,(2750) 
: 2750 — 1830 
17 460 \ = 1 Fe20 


1 — 0.9772 = 0.0228 


Wl 


Nl 


The probability that clouds are higher than 2750 m is therefore about 2.20 
percent if their behavior is as assumed. 


25 OTHER DISTRIBUTION AND DENSITY EXAMPLES 


Many distribution functions are important enough to have been given names. We 
give five examples. The first two are for discrete random variables; the remaining 
three are for continuous random variables. Other distributions are listed in 


Appendix F. 


Binomial 
LetO<p<t,andN= 1, 2,..., then the function 


a 
f= LD ( te = ip) *x— 4) (2.5-1) 


=0 


is called the binomial density function, The quantity (") is the binomial coefficient 


defined in (1.7-4) as 
NY cos, (2.5-2) 
k kN —k)! 


The binomial density can be applied to the Bernoulli trial experiment of Chapter 
|, It applics to many games of chance, detection problems in radar and sonar, 
and many experiments having only two possible outcomes on any given trial. 

By integration of (2.5-1), the binomial distribution function is found: 


F (x) = 5 (‘) KL — Xo Rul = k) (2.5-3) 
xX) = k P p ° . 


k=O 


Figure 2.5-1 illustrates the binomial density and distribution functions for 


N = 6and p = 0.25. 
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0.0330 
0.0044 0.0002 


g 1.0000 


0.9624 0.9954 0.999 
0.8306 


0.5 


9.1780 
5 : : Figure 2.5-1 Binomial density (a) and 
3 4 $ 6 xX distribution (b) functions for the case 
(b) N = 6 and p = 0.25, 
Poisson 


The Poissont random variable X has a density and distribution given by 


© xk. 
fhe eS R(x) (2.544) 
k=0 if 
ao b* 
Frye? Y ux h (2.55) 
k=Q “* 


where b > 0 is a real constant. When plotted, these functions appear quite similar 
to those for the binomial random variable (Figure 2.5-1). In fact, if N—» co and 
p— 0 for the binomial case in such a way that Np = b, a constant, the Poisson 
case results. : 

The Poisson random variable applies to a wide variety of counting-type 
applications. It describes the number of defective units in a sample taken from a 
production line, the number of telephone calls made during a period of time, the 


t After the French mathematician Siméon Denis Poisson (1781-1840). 
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number of electrons emitted from a small section of a cathode in a given time 
interval, etc. If the time interval of interest has duration T, and the events being 


counted are known to occur at an average rate 4 and have a Poisson distribu- : 


tion, then b in (2.5-4) is given by 
b=A1T (2.5-6) 


We illustrate these points by means of an example. 


vay 
eee 
Example 2.5-1 Assume automobile arrivals at a gasoline station are Poisson 
and occur at an average rate of 50/h. The station has only one gasoline 
pump. If all cars are assumed to require one minute to obtain fuel, what is 
the probability that a waiting line will occur at the pump? 

A waiting line will occur if two or more cars arrive in any one-minute 
interval, The probability of this event is one minus the probability that either 
none or one car arrives, From (2.5-6), with 4 = °°4, cars/minute and T = | 
minute, we have b = %, On using (2.5-5) 


Probability of a waiting line = 1 — F,(1) — F,(0) 
=l- eal + ;| = 0.2032 


We therefore expect a line at the pump about 20.32% of the time. 


ra 


Uniform 


The uniform probability density and distribution functions are defined by: 


— J 'Kb - a) asxsb 
Sx(x) = {6 ae eee cole 
0 x<a 
Fy(x)=((x—-a)(b—a)  asx<b (2.5-8) 
1 bsx 


for real constants — 09 <a < oo and b > a. Figure 2.5-2 illustrates the behavior 
of the above two functiohs. 

The uniform density finds a number of practical uses. A particularly impor- 
tant application is in the quantization of signal samples prior to encoding in 
digital communication systems. Quantization amounts to “rounding off” the 
actual sample to the nearest of a large number of discrete “ quantum levels,” The 
errors introduced in the round-off process are uniformly distributed. 
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Six) 


Ib - 4) 


(a) 


Figure 2.5-2 Uniform probubility density 
0 a b X function (a) and its distribution function 


(>) (b). 


Exponential 


The exponential density and distribution functions are: 


I pais ald x>a 
Sls) =< B® (2.5-9) 
0 x<a 


, b= ere « Sie 
Fix) = 0 x<a 


for real numbers ~0o <a <0o and b > 0. These functions are plotted in Figure 
2.5-3. - 

The exponential density is useful in describing raindrop sizes when a large 
number of rainstorm measurements are made. It is also known to approximately 
describe the fluctuations in signal strength received by radar from certain types of 
aircraft as illustrated by the following example, 


(2.5-10) 


i, 


Example 2.5-2 The power reflected from an aircraft of complicated shape 


that is received by a radar can be described by an exponential random vari- - 


able P, The density of P is therefore 


l 
— g7riPo p>o 
Selp) =% Po 
0 p<o 
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ey 


=-¢ 


b 


(a) 


| = eran 


0 a x Figure 2.5-3 Exponential density 
(b) (a) and distribution (b) functions. 


where Py is the average amount of received power. At some given time P 
may have a value different from its average value and we ask: what is the 
probability that the received power is larger than the power received on the 


average? 
We must find P{P > Po} = 1 - P{(P < Po} = 1 - F (Po). From (2.5-10) 


P{P > Po} =1-(l- eg” PolPo) = e~! ~ 0.368 
In other words, the received power is larger than its average value about 36.8 
per cent of the time. 


Rayleigh 
The Rayleight density and distribution functions are: 
2 
~ ty — ayer re 
payee ae aoe (2.5-11) 
0 x<a 
pee x2za 
Fy(x) = { (2.5-12) 
0 x<a 


t Named for the English physicist John William Steutl, Lord Rayleigh (1842-1919). 
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Figure 2.5-4 Rayleigh density (a) 
and distribution (h) functions. 


for real constants -—co <a< © and b> 0. These functions are plotted in 


Figure 2.5-4. 


The Rayleigh density describes the envelope of one type of noise when passed 
through a bandpass ‘filter. It also is important in analysis of errors in various 


measurement systems. 


2.6 CONDITIONAL DISTRIBUTION AND 


DENSITY FUNCTIONS 


The concept of conditional probability was introduced in Chapter 1, Recall that, 
for two events A and B where P(B) # 0, the conditional probability of A given B 


had occurred was 


P(A|B) = pA Se ¥ on ) (2.6-1) 


In this section we extend the conditional probability concept to include random 


variables. 


Sarees ey Amiserve VARIABLES, ANU KANDOM SIGNAL PRINCIPLES) 7 


Conditional Distribution 


Let A in (2.6-1) be identified as the event {X < x} for the random variatle XY. The’ 


resulting probability P{X < x|B} is defined as the conditional distribution func- 
tion of X, which we denote Fy (x|B). Thus 


P{X <x B} 


F(x] B) = P(X < x|B} = P(B) 


where we use the notation {X < xm B} to imply the joint event {X < x} qn B. 
This joint event consists of all outcomes s such that 


X(s) Sx and séeB (2.6-3) 


The conditional distribution (2.6-2 


) applies to discrete, continuous, or mixed 
random variables. 


Properties of Conditional Distribution 


All the properties of ordinary distributions apply to F,(x|B). In other words, it 
has the following characteristics: 


(1) F,(—co}B) =0 (2.6-4a) 
(2) Fy(co} B) = 1 (2.6-4b) 
(3) OS Fy(x}B) <1 (2.6-4c) 
(4) Fy(e{B)< Fy(xz|B) if xy < xy (2.6-4d) 
(5) Plxy <X < x,|B) = Fe(xz|B) — Fy(x, |B) (2.6-4e) 
(6) Fy(x* | B) = Fy(x| B) (2.6-4/) 


These characteristics have the same 


general meanings as described earlier follow- 
ing (2.2-2). ; 


Conditional Density 


In a manner similar to the ordinary density function, we define conditional density 

function of the random variable X as the derivative of the conditional distribution 
function. If we denote this density by /,(x| B), then 

dF ,(x| B) 

Sx(%| B) = a aa (2.6-5) 


If F,(x| B) contains step discontinuities, as when X is a discrete or mixed random 


variable, we assume that impulse functions are present in f(x] B) to account for 
the derivatives at the discontinuities. 


(2.6-2), 
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Properties of Conditional Density 


Because conditional density is related to conditional distribution He 
derivative, it satisfies the same properties as the ordinary density function. They 


are: 


(1) Sr(x]B) 20 (2.6-6a) 
(2) [" Sx(x| B) dx =1 (2.6-6b) 
(3) Fy(x| 8) = I. Sx(E| B) de Bs (2.6-6¢) 
(4) P{x,<X <$x,[B} = a Se(x| B) dx (2.6-6¢) 


We take an example (o illustrate conditional density and distribution. 


en a ee ee 


le 2.6-1 Two boxes have red, green, and blue balls in them; the num- 
Be at ball of each color is given in Table 2.6-1. Our experiment will be to 
select a box and then a ball from the selected box. One box (number a 
slightly larger than the other, causing it to be selected more dba ace . 
B, be the event “select the larger box” while B, is the event “select He 
smaller box.” Assume P(B,) = 49 and P(B2) = %4o.(B, and B, are mutua y 
exclusive and B, U B, is the certain event, since some box must be sclected; 
therefore, P(B,) + P(B,) must equal unity.) 
Now define a discrete random variable X to have values x, = 1, x, = 2, 
and x, = 3 when a red, green, or blue ball is selected, and let B be an event 
equal to cither B, or B,. From Table 2.6-1: 


5 _ 80 
P(X = 1|B = Bi) = 755 P(X = 1B = By) =7 5 
35 7 _ 60 
P(X = 2|B = By) = 755 P(X = 2|B = B,) = 75 
60 : _ 10 
P(X = 3|B= Bi) =75. P(X = 3[B = B= 7 


Table 2.6-1 Numbers of colored 
balls in two boxes 


Box 
x; Ball color 1 2 Totals 
J Red 5 80 85 
2 Green 35 60 95 
3 Blue 60 10 70 
Totals 100 150 250 
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The conditional probability density {x(x | B,) becomes 
5 35 60 
Sfx By) = 100 d(x — 1) + 100 5(x — 2) + 100 5(x — 3) 


By direct integration of f(x | By): 


5 35 60 
F y(x| By) = 75H * ~ 1) + 799 u(x — 2) + == u(x — 3) 


100 


Hy (x) of Py (x18 ,) 
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Figure 2.6-1 Distributions(a)and den- 
sities (b) and (c) applicable to Pixnm- 
ple 2.6-1. 
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For comparison, we may find the density and distribution of X by deter- 
mining the probabilities P(X = 1), P(X = 2), and P(X = 3). These are found 
from the total probability theorem embodied in (1.4-10): 


P(X = 1) = P(X = 1|B,)P(B,) + P(X = 1 | B,)P(B2) 


5 2 80 (8 
= 100 (2) + 156 (=) eee 


Thus 
Sx() = 0.437 d(x — 1) + 0.390 d(x — 2) + 0.173 d(x — 3) 


and 
F x(x) = 0.437u(x — 1) + 0.390u(x — 2) + 0.173u(x — 3) 


These distributions and densities are plotted in Figure 2.6-1. 


*Methods of Defining Conditioning Event 


The preceding example illustrates how the conditioning event B can be defined 


from some characteristic of the physical experiment. There are several other ways 
of defining B (Cooper and McGillem, 1971, p. 61). We shall consider two of these 


in detail. 
In one method, event B is defined _in terms of the random variable X. We 
discuss this case further in the next paragraph. In another method, event_B may 


depend on some random variable other than X, We discuss this case further in 
Chapter 4. ; 
One way to define event B in terms of X is to let 
B={X <b} (2.6-7) 


where b is some real number —0o < b < oo. After substituting (2.6-7) in (2.6-2), 


we gett 
_ P(X sxaX sb (2.6-8) 


Fy(x|X <b) = P(X Sx1X sd} Fx s3) 


+ Notation used has allowed for deletion of some braces for convenience. Thus, Fy(x]{X < 6}) is 
written Fix] X <b) and PUX Ss x) a (X s h}) becomes P(X sx aX sh}. 
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for all events {X <b} for which P{X < b} 40. Two cases must be considered; 
one is where b < x; the second is where x <b. If b $ x, the event {x <b) is a 
subset of Ihe even’ {XS x}, 0 TX Sx) n (X < b} = (X sb}. Equation (2.6-8) 
becomes 


P{X sxnXsb}_ P{X <b} 


Fy{x|X <b) == P(X <5) = B(x <b} 7 bsx  (2.6-9) 


When x<b the event {X <x} is a subset of the event {X <5}, so é 


{X <x} 7 {X < b} = (X < x} and (2.6-8) becomes 
P{X sxnXsb}_ P{X sx} _ Fryx) 


FY(x|X <b) = —— 6- 
x(x |X < b) P(X Sb} P(X <5} ” Fy(b) x<b (2.6-10) 
By combining the last two expressions, we obtain i 
F(x) 
eer <b 
Fy(x|X <b)={ Fx) ~ (2.6-11) 
1 bsx 
Fy(xlX¥ <8) or Fy(x) 


Fy(x) 


FylxlX < 6) 


0 
(a) 


Sn (XIX <b) or fy (x) 


Sxl X <b) 
Sx) 


0 6 x 
(b) 


Figure 2.6-2 Possible distribution functions (a) and density functions (b) applicable to a conditioning 
event B= {X <b}. 
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The conditional density function derives from the derivative of (2.6-11): 


AOL), 
fees X <b) = {Fub) PES Kode (2.6-12) 
0 x2b 


Figure 2.6-2 sketches possible functions representing (2.6-11) and (2.6-12). 
From our assumption that the conditioning event has nonzero probability, 
we have 0 < F,(b) < 1, so the expression of (2.6-11) shows that the conditional 
distribution function is never smaller than the ordinary distribution function: 
Fy(x|X <b) 2 F x(x) (2.6-13) 


A_similar statement holds for the conditional density function of (2.6-12) 


wherever it is nonzero: 
Sx(x |X <b) = f(x) x<b  (2,6-14) 


The principal results (2.6-11) and (2,6-12) can readily be extended (o the more 
general event B = {a < X < b} (sce Problem 2-39). 


Example 2.6-2 The radial “ miss-distance” of landings from parachuting sky 
divers, as measured from a target’s center, is a Rayleigh random variable 
wilh b = 800 m? and a = 0. From (2.5-12) we have 


F(x) = LE — 777899 }u(x) 
The target is a circle of 50-m radius with a bull’s eye of 10-m radius, We find 
She probability of a parachuter hitting the bull’s cye given that the landing is 
on the target. 
The required probability is given by (2.6-11) with x = 10 and b = 50: 
P(bull’s cye [landing on target) = Fy(10)/F (50) 
= (I —~¢7 1O0/HOO) 1 | = GPA BOINO) = 0.1229 


Parachuter accuracy is such that about 12.29% of landings falling on the 
target will actually hit the bull's cyc. 


PROBLEMS 


2-1 The sample space for an experiment is S = {0,-1, 2.5, 6}. List all possible 
values of the following random variables: 

(a) X = 2s 

(b) X = 5s? — 1 

(c) X = cos (xs) 

(d) X =(1 —3s)7! 
2-2 Work Problem 2-1 for S = {-2 <s < 5}. 
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2-3 Given that a random variable X has the following possible values, statc if X 
is discrete, continuous, or mixed. 

(a) {-20 <x < —5} 

(b) (10, 12<xs 14, 15, 17} 

(c) (10 fors > 2and Sforss 2, where | <s < 6} 

(d) (4, 3.1, 1, —2} 


2-4 A random variable X ‘is a function. So is probability P. Recall that the 
domain of a function is the set of values its argument may take on while its range 
is the sct of corresponding values of the function. In terms of sels, events, and 
sample spaces, state the domain and range for X and P. 


2.5 A man matches coin flips with a friend. He wins $2 if coins match and loses 
$2 if they do not match. Sketch a sample space showing possible outcomes for 
this experiment and illustrate how the points map onto the real line x that defines 
the values of the random variable X = "dollars won on a trial.” Show a second 
mapping for a random variable Y = “dollars won by the friend on a trial.” 


2-6 Temperature in a given city varies randomly during any year from —21 to 
49°C. A house in the cily has a thermostat that assumes only three positions: 1 
represents “call for heat below 18.3°C," 2 represents “ dead or idle zone,” and 3 
represents “call for air conditioning above 21.7°C." Draw a sample space for 


this problem showing the mapping necessary to define a random variable 
X = “thermostat setting.” 


2-7 A random voltage can have any value defined by the set S={asss by. A 
quantizer divides S into 6 equal-sized contiguous subsets and gencrates a voltage 
random variable X having values {—4, -2,0, 2,4, 6}. Each value of X is equal 
to the midpoint of the subset of S from which it is mapped. 

(a) Sketch the sample space and the mapping to the line x that defines the 
values of X. 

(b) Find a and b. 

*9.8 A random signal can have any voltage value (at a given time) defined by the 
set S= {tig <SS ay}, where ao and ay are real numbers and N is any integer 
N > 1. A voltage quantizer divides S into N equal-sized contiguous subsets and 
converts the signal level into one of a set of discrete levels a,, = 1,2,.06 N, that 
correspond to the “input” subsets {a,.1 <5 S4,}. The set (ay, @yy +-+y Gy} can 
be taken as the discrete values of an “output” random variable X of the quan- 
tizer. If the smallest “input” subset is defined by 4 = a1 — 4o and other subsets 
by a, — d,-4 = 2"" 1A, determine A and the quantizer levels a, in terms of dg, Qn, 
and N, 

2-9 An honest coin is tossed three times. 

(a) Sketch the applicable sample space S showing all possible clements. 
Let X be a random variable that has values representing the number of 
heads obtained on any (riple toss. Sketch the mapping of S onto the real linc 
defining x. 

(b) Find the probabilities of the values of X. 


2-10 Work Problem 2-9 fora biased coin for which P{head} = 0.6. 
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2-11 Resistor R, in Figure P2-11 is randomly selected from a box of resistors 
containing 180-Q, 470-Q, 1000-M, and 2200-Q resistors. All resistor values have 
the same likelihood of being selected. The voltage E, is a discrete random vari- 


a « able. Find the set of values E, can have and give their probabilities. 


R, = 8202 


E, i 


Figure P2-11 


2-12 Bolts made on a production line are nominally designed to have a 760-mm 
length. A go-no-go testing device eliminates all bolts less than 650 mm and over 
920 mm in length. The surviving bolts are then made available for sale and their 
lengths are known to be described by a uniform probability density function. 
A certain buyer orders all bolts that can be produced with a +5% tolerance 
about the nominal length. What fraction of the production line's output is he 
purchasing? 
2-13 Find and sketch the density and distribution functions for the random vari- 
ables of parts (a), (b), and (c) in Problem 2-1 if the sample space elements have 
equal likclihoods of occurrence, 
2-14 If temperature in Problem 2-6 is uniformly distributed, sketch the density 
and distribution functions of the random variable X. 
2-15 For the uniform random variable defined by (2.5-7) find: 

(a) P{0.9a + 0.1b <X S0.7a + 0.3b} 

(b) P{(a+ b/2<XS b} 
2-16 Determine which of the following are valid distribution functions: 


1 —e7*!? x20 
(a) Gy{x) = 
; x <0 
oO : x<0 
(0) Gfx) = {0.5 + 05 sin [x(x — 1/2] Osx <2 


1 x22 
: : 
(c) Gy(x) = a (u(x — a) — u(x — 2a)] 
2-17 Determine the real constant a, for arbitrary real constants m and 0 <b, 


such that 
Silx) = gent 


is a valid density function (called the Laplacet density). 


t After the French mathematician Marquis Pierre Simon de Laplace (1 749-1827). 
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2-18 An intercom system master station provides music to six hospital rooms. 


The probability that any one room will be switched on and draw power at any: 


time is 0.4. When on, a room draws 0.5 W. 

(a) Find and plot the density and distribution functions for the random vari- 
able “ power delivercd by the master station.” 

(5) If the master-station amplifier is overloaded when more than 2 W is 
demanded, what is its probability of overload? 


*2-19 The amplifier in the master station of Problem 2-18 is replaced by a 4-W : 
unit that must now supply 12 rooms. Is the probability of overload better than if 4 


two independent 2-W units supplied six rooms each? 
2-20 Justify that a distribution function Fy(x) satisfies (2.2-2a, b, c). 


2-21 Use the definition of the impulse function to evaluate the following 
integrals. ! 
(Hint: Refer to Appendix A.) 


4 
(a) (3x? + 2x — 4)5(x — 3.2) dx 
3 


(b) c cos (62x)d(x ~ 1) dx 


@ 248(x — 2) dx 
(©) ie x4 + 3x? +2 


(d)t ie d(x — xXo)e~4"* dx 


3 
(e) { u(x ~ 2)5(x — 3) dx 
-3 


2-22 Show that the properties of a density function f,(x), as given by (2.3-6), arc 
valid. 
2-23 For the random variable defined in Example 2.3-1, find: 


(a) P{xo ~ 0.6a < X < x9 + 0.3a} 
(b) P{X = xo} 


2-24 A random variable X is gaussian with ay = Oand oy = |. 
(a) What is the probability that |X| > 2? 
(b) What is the probability that X > 2? 


2-25 Work Problem 2-24 if ay = 4and oy = 2. 
_ 2-26 For the gaussian density function of (2.4-1), show that 


i Xf(x) dx = ax 


“ow 


t The quantity / is the unit-imuginary; that is, J = /— 1. 
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2-27 For the gaussian density function of (2.4-1), show that 
| (x — ax)?fy(x) dx = 0% 
-@ 


2-28 A production line manufactures 1000-Q resistors that must satisfy a 10% 
tolerance, 

(a) If resistance is adequately described by a gaussian random variable X for 
which ay = 1000 Nand ay = 40 Q, what fraction of the resistors is expected to be 
rejected? 

(b) if a machine is not properly adjusted, the product resistancés change lo 

the case where ay = 1050 (5% shift). What fraction is now rejected? 
2-29 Cannon shell impact position, as measured along the line of fire from the 
target point, can be described by a gaussian random variable X, It is found that 
15.15% of shells fall 11.2 m or farther from the target in a direction toward the 
cannon, while 5.05% fall farther than 95.6 m beyond the target..What are ay and 
ox for X? 


2-30 (a) Use the exponential density of (2.5-9) and solve for J, defined by. 


I, = { x?fy(x) dx 
(b) Solve for 1, defined by 


I, _ t xfy(x) dx 


(c) Verify that J, and J, satisfy the equation J, ~ 1? = b?, 
2-31 Verify that the maximum value of f,(x) for the Rayleigh density function of 
(2.5-11) occurs atx =a+ /b/2 and is equal to J2)b exp (— '4) = 0.607,/2/b. 
This value of x is called the mode of the random variable. (In general, a random 
variable may have more than one such value—explain.) 
2-32 Find the value x =x, of a Rayleigh random variable for which 
P{X < Xo} = P{xq < X}, This value of x is called the median of the random vari- 
able. 
2-33 The lifetime of a system expressed in weeks is a Rayleigh random variable 
X for which 

(x/200)e777/499 Os x 


Sxlx) = o x<0 


(a) What is the probability that the system will not last a full week? 
(6) What is the probability the system lifetime will exceed one year? 
2-34 The Cauchyt random variable has the probability density function 


- b/x 
Sula) = b? +(x — a)? 


t After the French mathematician Augustin Louis Cauchy (1789-1857). 
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for real numbers 0 < band -—co <a<o. Show that the distribution function of 


X is 
‘ 1 t -,{x-4 
Fy(x) = 5 + = tan ( 7 ) 


2-35 The Log-Normal density function is given by 


exp {—[In (x — b) — ax]?/203} 


Si(x) = JSinax(x — b) 
0 x<b 


x2b 


for real constants 0<¢y, —© <ay<oo, and —-~a <b< 0, where In (x) 
denotes the natural logarithm of x. Show that the corresponding distribution 


function is 
In (x — b) - 2) 
Fi ——_—__—_— x2>b 
F(x) = | ox 
0 x<b 


where F(+) is given by (2.4-3). F 
2-36 A random variable X is known to be Poisson wilh b = 4. 
(a) Plot the density and distribution functions for this random variable. 
(b) What is the probability of the event {0< X <5}? 
2-37 The number of cars arriving at a certain bank drive-in window during any 
10-min period is a Poisson random variable X with b = 2. Find: 
(a) The probability that more than 3 cars will arrive during any 10-min 
period. 
((b) The probability that no cars will arrive. 
2-38 Rework Example 2.6-1 to find f,(x| B2) and F,(x|B,). Sketch the two 
functions, 
*9.39 Extend the analysis of the text, that leads to (2.6-11) and (2.6-12), to the 
more general event B = {a< X < }}. Specifically, show that now 


0 x<a 
F(x) — Fx(@) 

_ (4 b 

Fy(xja<X sb) F ,(b) = F x(a) agx< 
1 bsx 
and 
0 x<a 
Sx(x) Asi f(x) aexeb 


fisla<X $= Vay Rg SiG 
0 bsx 
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*2.40 Consider the system having a lifetime defined by the random variable X in 
Problem 2-33. Given that the system will survive beyond 20 weeks, find the prob- 
ability that it will survive beyond 26 wecks. 


ADDITIONAL PROBLEMS 


2-41 A sample space is defined by S = {1,2 <5 53,4, 5}. A random variable is 
defined by: X =2 for 0<s<2.5, X =3 for 25<s<35, and X =5 for 
35<55 6. 

(a) Is X discrete, continuous, or mixed? 

(b) Give a set that defines the values X can have. 

2-42 A gambler flips a fair coin three times. 

(a) Draw a sample space S for this experiment. A random variable X rep- 
resenting his winnings is defined as follows: He loses $1 if he gets no heads in 
three flips; he wins $1, $2, and $3 if he obtains 1, 2, or 3 heads, respectively. Show 
how elements of S map to values of X. 

(b) What are the probabilities of the various values of X? 

2-43 A function Gy(x) = a[1 + (2/n) sin™' (x/c)] rect (x/2c) + (4 + b)u(x — c) is 
defined for all —co <x < 00, where c > 0, b, and a are real constants and rect (-) 
is defined by (E-2). Find any conditions on a, b, and c that will make G(x) a 
valid probability distribution function, Discuss what choices of constants corre- 
spond to a continuous, discrete, or mixed random variable. 

2-44 (a) Generalize Problem 2-16(a) by finding values of real constants a and b 
such that 


G(x) = [1 — a exp (—x/b))u(x) 
is a valid distribution function. ; 
(b) Are there any values of a and b such that G(x) corresponds to a mixed 

random variable X? . 
2-45 Find a constant b > 0 so that the function 

F e*/4 O<xsb 
y Sx) = ‘6 sth ees 
is a valid probability density. 
2-46 Given the function 

- x(x) = 4 cos (%x/2b) rect (x/2b) 


find a value of b so that gx(x) is a valid probability density. 
2-47 A random variable X has the density function 
Salx) = (‘h)u(), exp (—x/2) 
Define events A = {1 <X <3}, B= {X $25}, and C=A 2 B. Find the prob- 
abilities of events (a) A, (b) B, and (c) C. 
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2-57 A computer undergoes down-time if a certain critical component fails. This 
component is known to fail at an average rate of once per four wecks. No signili- 
cant down-time occurs if replacement components are on hand because repair 
can be made rapidly. There are three components on hand and ordered replace- 


ments are not due for six weeks. 7 
(a) What is the probability of significant down-time occurring before the ‘ 


*2-48 Let (x) be a continuous, but otherwise arbitrary real function, and let «a 
and b be real constants. Find G(a, 6) defined by 


G(a, b) = [° (x) d(ax + b) dx 


pa 


(Hint: Use the definition of the impulse function.) 


\, 


2-49 For real constants b > 0, c > 0, and any a, find a condition on constant a 
and a relationship between c and a (for given b) such that the function 


a{l — (x/b Osxse 
fete) = { [1 ~ (x/b)] 
0 elsewhere 
is a valid probability density, 
2-50 A gaussian random variable X has ay = 2,and oy = 2.: 
(a) Find P{X > 1.0}. 
(b) Find P{X < —1.0}. 
2-51 In a certain “junior” olympics, javelin throw distances are well approx- 
imated by a gaussian distribution for which ay = 30 m and oxy =5m. Inu quali- 
fying round, contestants must throw farther than 26 m to qualify. In the main 
event the record throw is 42 m. 
(a) What is the probability of being disqualified in the qualifying round? 
(5) In the main event what is the probability the record will be broken? 
2-52 Suppose height to the bottom of clouds is a gaussian random variable X for 
which ay = 4000 m, and ay = 1000 m. A person bets that cloud height tomorrow 
will fall in the set A = {1000 m < X < 3300 m} while a second person bets that 
height will be satisfied by B = {2000 m < X < 4200 m}. A third person bets they 
are both correct. Find the probabilities that each person will win the bet. 


2-53 Let X be a Rayleigh random variable with a = 0, Find the probability that 
X will have values larger than its mode (see Problem 2-31). 
2-54 A certain large cily averages three murders per week and their occurrences 
follow a Poisson distribution. ; 

(a) What is the probability that there will be five or more murders in a given 
week? 

(b) On the average, how many weeks a year can this city expect to have no 
murders? 

(c) How many weeks per year (average) can the city expect the number of 
murders per week to equal or exceed the average number per week? 


* 2-55 A certain military radar is set up at a remote site with no repair facilities. If 
the radar is known to have a mean-time-between-failures (MTBF) of 200 h find 
the probability that (he radar is still in operation one week later when picked up 
for maintenance and repairs. 

2-56 If the radar of Problem 2-55-is permanently located at the remote site, find 
the probability that it will be operational as a function of time since its sct up. 


ie-ercsbihd yor tot to 
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ordered components arrive? 7 
(b) If the shipment is delayed two weeks what is the probability of significant 
down-time occurring before the shipment arrives? : 

*2.58 Assume the lifetime of a laboratory research animal is defined by a Rayleigh 
density with a = 0 and b = 30 weeks in (2.5-11) and (2.5-12). If for some clinical 
reasons it is known that the animal will live at most 20 weeks, what is the prob- 
ability it will live 10 weeks or less? 

*2.59 Suppose the depth of water, measured in meters, behind a dum is described 
by an exponential random variable having a density 


Sx) = (1/13.5) exp (—x/13.5) 


There is an emergency overflow at the (op of the dam that prevents the depth 
from excecding 40,6 m. There is a pipe placed 32.0 m below the overflow (ignore 
the pipe’s finite diameter) that feeds water to a hydroelectric generator. 

(a) What is the probability that water is wasted through emergency over- 
flow? 

(b) Given that water is not wasted in overflow, what is the probability the 
generator will have water to drive it? 

(c) What is the probability that water will be too low to produce power ? 

*2.60 In Problem 2-59 find and sketch the distribution and density functions of 
water depth given that water will be deep enough to generale power bul no water 
is wasled by emergency overflow. Also sketch for comparisons (he distribution 
and density of water depth without any conditions? 

*2.61 In Example 2.6-2 a parachuter is an “expert” if he hits the bull's eye. If he 
falls outside the bull's eye but within a circle of 25-m radius he is called 
“qualified” for competition, Given that a parachuter is not an expert but hits the 
target what is the probability of being “ qualified?” 
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OPERATIONS ON 
ONE RANDOM. VARIABLE—EXPECTATION 


3.0 INTRODUCTION 


The random variable was introduced in Chapter 2 as a means of providing a sys- 
tematic definition of events defined on a sample space. Specifically, it formed a 
mathematical model for describing characteristics of some real, physical world 
random phenomenon. In this chapter we extend our work to include some 
important operations that may be performed on a random variable, Most of these 
operations are based ona single concept—expectation. 


3.4. EXPECTATION 


Expectation is the name given to the process of averaging when a random vari- 
able is involved. For a random variable X, we use the notation E[X], which may 
be read “the mathematical expectation of X," “ the expected value of X,” “the 
mean valuc of X,” or “the statistical average of X.” Occasionally we also use the 
notation X which is read the same way as E[X]; that is, X = E[X).7 

Nearly everyone is familiar with averaging procedures. An example that 
serves (0 tic a familiar problem to the new concept of expectation may be the 
casiest way to proceed. 


t Up to this point in this book an overbar represented the complement of a set or event, Hence- 
forth, unless specifically stated otherwise, the overbar will always represent a mean value. 
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Example 3.1-1 Ninety people are randomly selected and the fractional dollar 

yalue of coins in their pockets is counted. If the count goes above a dollar, 

the dollar value is discarded and only the portion from 0¢ Lo 99¢ is accepted, 

It ig found that 8, 12, 28, 22, 15, and 5 people had !8¢, AS¢, 64¢, 72¢, 776, and 

95¢ in their pockets, respectively. \ 
Our everyday experiences indicate that the average of these values is 


8 12 28 22 
Average $ = o19( =) + aas( 2) + asa( 2) + on(22) 


15 s) 
+ an() + ass( =) 
x $0.632 


eae 


Expected Value of a Random Variable 


The everyday averaging procedure used in the above example carries over 
directly to random variables. In fact, if X is the discrete random variable 
“fractional dollar value of pocket coins,” it has 100 discrete values x; that occur 
with probabilities P(x,), and its expected value E[X] is found in the same way as 


in the example: - 
100 aie 


ELX] = ¥ x P(x) (3.1-1) 
The values x, identify with the fractional dollar values in the example, while P(x) 
is identified with the ratio of the number of people for the given dollar value to 
the total number of people. If a large number of people had been used in the 
“sample” of the example, all fractional dollar values would have shown up and 
the ratios would have approached P(x;). Thus, the average in the example would 
have become more like (3.1-1) for many more than 90 people. 

In general, the expected value of any random variable X is defined by 


atx) = 8 =| 


2 


Xfxlx) dx (3.1-2) 


If X happens to be discrete with N possible values x, having probabilities P(x,) of 
occurrence, then ; 


N 
Sx) = 2% Pedi — x) (3.1-3) 


iy 


from (2.3-5). Upon substitution of (3.1-3) into (3.1-2), we have 


N 

ELX] = YxiP(x) discrete random variable (3.1-4) 
t=4 

Hence, (3.1-1) is a special case of (3.1-4) when N= 100. For some discrete 

random variables, N may be infinite in (3.1-3) and (3.1-4). 
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-_o eee 


Example 3,1-2 We determine the mean value of the continuous, exponen- 
tially distributed random variable for which (2.5-9) applies: 
I 


—(x-a)/b 
~e x>a 
Sxl) = (bd 


10 x<a 
From (3.1-2) and an integral from Appendix C: | - 


2 a/b «2 
E[X] -| ore iis -= xe Aveadd 


i ee 


If a random variable's density is symmetrical about a line x =a, then 
ELX] = a: that is, : 


E[X]=a if f(x+a)=fy(—-x+ a) (3.1-5) 


Expected Value of a Function of a Random Variable 


As will be evident in the next section, many useful parameters relating to a 
random variable X can be derived by finding the expected value of a real func- 
tion g(+) of X. Tecan be shown (Papoulis, 1965, p. 142) that this expeeted value is 
given by 
wo 
‘ E{y(X)] -| Ox) Sx(x) dx (3.1-6) 
ox.) 


If X is a discrete random variable, (3.1-3) applies and (3.1-6) reduces to 


N 
E(g(X)] = ¥ o(x)P(x)) discrete random variable (3.1-7) 
ia} 
where N may be infinite for some random variables. 


ee 


Example 3.1-3 It is known that a particular random voltage can be rep- 
resented as a Rayleigh random variable V having a density function given by 
(2.5-11) with a = 0 and b = S, The voltage is applied to a device that gener- 
ates a vollage Y = g(V) = V? (hat is equal, numerically, to the power in V (in 
a 1- resistor). We find the average power in V by means of (3.1-6): 


co) 3 
Power in V = E[g(V)] = E[V?] = { = e725 dy 
0 
By letting € = v?/5, dé = 2v dv/5, we obtain 


Power in y=5| Eée§dé=S5W 
0 
after using (C-46). 
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*Conditional Expected Value 


If, in (3.1-2), f,(x) is replaced by the conditional density /y(x |B), where B is any 
event defined on the sample space, we have the conditional expected value of X, 
denoted E[X | B]: 


EX |B] = { : xf¢(x| B) dx (3.1-8) 


One way to define event B, as shown in Chapter 2, is to let it depend on the 
random variable X by defining 


B={X <b} -0o<b<0 (3.1-9) 
We showed there that 
Sx(x) 5 
flxIX<d)={Po foodx ** (3.1-10) 
0 x2b 


Thus, by substituting (3.1-10) into (3.1-8): 


ie Xfy(x) dl 
ETN |X < b7 = “in xfx(x) ax 


Shs Sex) dx 


which is the mean value of X when X is constrained to the set {X < bh). 


Gatehly 


3.22 MOMENTS 


An immediate application of the expected value of a function g(-) of a random 
variable X is in calculating moments. Two types of moments are of interest, those 
about the origin and those about the mean. 


Moments About the Origin 


The function 


g(X)=X" n=O, 1,2)... (3.2-1) 


when used in (3.1-6) gives the moments about the origin of the random variable 
X. Denote the nth moment by m,. Then, 


a 


m, = ELX"} = { x"fy(x) dx | (3.2-2) 


“oO 


Clearly my = 1, the area of the function fy(x), while m, = X, the expected valuc 
of X. 
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after using an integral from Appendix C. However, from Example 3.1-2, X= 


Central Moments 
E[X] = (a + b), so 


Moments about the mean valuc of X are called central moments and are given the A ot = b? 
symbol ji,. They are defined as the expected value of the function eS! 
2 : The reader may wish to verily this result by finding the second moment 
g(X) = (X — XY n=O, 1,2,.-- (3.2-3) E[X?] and using (3.2-6). 
which is 
ae 0 ave The third central moment jt, = E((X — X)°] is a measure of the asymmetry 
ty = EX — XY") = [ & — X)"fxlx) dx (3.2-4) of f,(x) aboul x = X = m,. It will be called the skew of the density function. Ifa 


density is symmetric about x = X, it has zero skew. In fact, for this case jt, = 0 
for all odd values of n. (Why?) The normalized third central moment ji,/oX is 
known as the skewness of the density function, or, alternatively, as the coefficient 
of skewness. 


The moment ftp = I, the area of fy(x), while wy = 0. (Why?) 


Variance and Skew 


The second central moment jy is so important we shall give it the name variance 


and the special notation o2. Thus, variance is given byt Example 3.2-2 We continue Example 3.2-1 and compute the skew and cocffi- 


cient of skewness for the exponential density. From (3.2-4) with n= 3 we 
have 


ah pty = EN — Xy = (" (x — X)Pfy(x) dx (3.2-5) 


“am 


The positive square rool oy of variance is called the standard deviation of X; it is 
4 measure of the spread in the function fy(x) about the mean. 
Variance can be found from a knowledge of first and second moments. By 


expanding (3.2-5), we have} 
ah = ELX? — RX + X27) = ELX7] — 2RELX] + X? 
= E[X?)- X? =m, - mi? (3.2-6) 


Next, we have 


as o 3 
ree | xo goa dx = at + 3a2h + Gab? + 6b? 


after using (C-48). On substituting KX =atb and of = b? from the earlier 
example, and reducing the algebra we find 


Se 


Example 3.2-1. Let X have the exponential density function given in Example fly = 2b 
3.1-2, By substitution into (3.2-5), the variance of X is ; , 
3 
ps2 
ox 


@o 
oie 
ak = | (x - xy em en iscah dx 
o b This density has a relatively large coefficient of skewness, as can be secn intu- 
By making the change of variable ¢ = x ~ € wedbiain a. 
eT kab o j : | 
oe | ge 8h dE = (at b— 2) +B? 
-* 


*3,3 FUNCTIONS THAT GIVE MOMENTS 
Two functions can be defined that allow moments lo be calculated for a random 


variable X. They are the characteristic function and the moment gencrating 
function. 


i , 
t ‘The subscript indicates that @3 is the variance of a random variable X. For a random variable Y 


its variance would be al, 
t We use the fact that the expected value of a sum of functions of X equals the sum of expected 
values of individual functions, as the reader can readily verify as an exercise, 
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* Characteristic Function 
The characteristic function of a random variable X is defined by 


®,(w) = E[e**] (3.3-1) 


where J = /—1. It is a function of the real number —0o <w < oo. If (3.3-1) is 
written in terms of the density function, ®y(w) is seen to be the Fourier transformt 
(with the sign of w reversed) of /,(x): 


@,{w) = [ Silxel®* dx (3.3-2) 


Because of this fact, if &,(w) is known, fy(x) can be found from the inverse Fourier 
transform (with sign of x reversed) 


: a aa 
Jy(x) = = { DP y(cwo)e 7 4* leo (33-3) 
By formal differentiation of (3.3-2) n times with respect to w and setting w = 0 in 
the derivative, we may show that the nth moment of X is given by 


a" (w) 


mn, = (-)" dw" 


(3.3-4) 


w=0 


A major advantage of using ©,(w) to find moments is that ®,(w) always 
exists (Davenport, 1970, p. 426), so the moments can always be found if ©,(w) is 
known, provided, of course, the derivatives of ®(w) exist. 


Tt can be shown that the maximum magnitude of a characteristic function is 
unity and occurs at w = 0; that is, 


[D,(w)| < (0) = 1 (3.3-5) 
(See Problem 3-24.) 


Example 3.3-1 Again we consider the random variable with the exponential 


density of Example 3.1-2 and find its characteristic function and first 
moment. 


t Readers unfumiliar with Fourier transforms should interpret ©,(w) as simply the expected value 


of the function g(X) = exp (jwX). Appendix D is included as a review for others wishing tc refresh 
their background in Fourier transform theory, 
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By substituting the density finction into (3,3-2), we get 


ay eh o eee 
dy (cw) ee a ge bk ailbydox dx = e dy 
, 0 » J, 


Evaluation of the integral follows the use of an integral from Appendix C: 


etl? 7 ba sole w 
i baa Gres ‘| 
ein ss 
Pipe job 


The derivative of (cv) is 


dD (0) ual —_ fe ib | 
piel he ery 5 
dw 1—jwb (1 —jwb) 


so the first moment becomes 


mn, (-/) dD (cw) 


=a +b, 


dw |yao 


in agreement with mm, found in Example 3.1-2. 


mn 


*Moment Generating Function 


Another statistical average closely related to the characteristic function is the 
moment generating function, defined by 


M x(v) = Efe"*] (3.3-6) 
where v is a real number —co < v < 09. Thus, M,(v) is given by 


M,(v) = in Sx(xye* dx (3.3-7) 


The main advantage of the moment generating function derives from ils 
ability to give (he moments. Moments are related to M,(v) by the expression: 


(3.3-8) 


v0 


The main disadvantage of the moment generating function, as opposed to the 
characteristic function, is that it may not exist for all random variables. In fact, 
M,(v) exists only if all the moments exist (Davenport and Root, 1958, p. 52). 
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Example 3.3-2 To illustrate the calculation and use of the moment gencr- 


ating function, let us reconsider the exponential density of the carlier cxam- |. ie 


ples. On use of (3.3-7) we have 


Mx) 


u 


i 


oy 
| Ze gen abyy® dx 
, oD 


el wo 
as ply ims dx 
ae 


In evaluating M(v) we have used an integral from Appendix C. 
By differentiation we have the first moment 


m= 


dy 


: e™ {a(t — by) + bj 


in| 
v=0 


=at+b 


(1 — bv)? a0 


which, of course, is the same as previously found. 


3.4 TRANSFORMATIONS OF A RANDOM VARIABLE 
j 


‘ ; | 
Quite often one may wish to transform (change) 
fa transformation 


new random variable Y by means 0 


Typically, the density function f,() 


Y¥ = T(X) 


or distribution function F(x) of X is known, 
and the problem is to determine either the density function Srly) oF distribution 
function Fy(y) of Y. The problem can be viewed as a “ black box ” with input X, 


output Y, and “ transfer characteristic” Y = T(X), as illustrated in Figure 3.4-1. 


In-general, X can be a discrete, continuous, or a mixed random variable. In 
turn, the transformation T can be linear, nonlinear, segmented, staircase, cle. 
Clearly, there are many cases to consider in a general study, depending on the 
form of X and T. In this section we shall consider only three cases: (1) X contin- 
uous and T continuous and cither monotonically increasing or decreasing with 


X; (2) X continuous and T continuous but nonmonotonic; (3) X discrete and T 


x y=7(X) Y 


fy) fry) 


Figure 3.4-1 Transformation of a random variable X 


lo a new random variable Y. 


one random variable X into a 


(3.4-1) 
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continuous. Note that the transformation in all three cases is assumed contin- 
uous. The concepts introduced in these three situations are broad enough that 
the reader should have no difficulty in extending them to other cases (sce 
Problem 3-32). 


Monotonic Transformations of a Continuous Random Variable 


A transformation T is called monotonically increasing if T(x,) < T(x) for any 
X, <X2- itis monotonically decreasing if T(x,) > T(x4) for any Xx, <X2+ 

Consider first the increasing tr: ation, We assume that 7" is continuous 
and differentiable at all values of x for which fx(x) # 0, Let Y have a particular 
value Yo corresponding to the particular value Xp of X as shown in Figure 3.4-2a. 
The two numbers are related by my 


Xo = T™'(Yo) (3.4-2) 


where T”! represents the inverse of the transformation T. Now the probability 
of the event {Y < yo} must equal the probability of the event {X < Xo} because 
of the one-to-one correspondence between X and Y. Thus, 


Fy(yo) = PLY S Yo} = P{X S$ Xo} = F x(Xo) 


Yo = T(Xo) or 


(3.4-3) 


Figure 3.4-2 Monotonic trans- 
formations: (a) increasing, and (6) 
decreasing. [Adapted from Peebles 
(1976) with permission of publi- 
shers Addison-Wesley, Advanced 
Book Program.} 


‘ 


\ 


or 


yo xo T~'(yo) 
[ Sr) dy = { Sx(x) dx (3.4-4) 


-, —-@ 


Next, we differentiate both sides of (3.4-4) with respect to yy using Leibniz’s rulet 
to get 


five) = fal T-"yol] eet 34-5) 
Since this result applies for any yo, we may now dep the subscript and wrile 
feast ZO (3.4-6 
or, more eonipacily: ; 
fu) = fo (3.4-7) 


In (3.4-7) it is understood that x is a function of y through (3.4-2). 


: A consideration of Figure 3.4-2b for the decreasing transformation verifies 
that 


Fy(yo) = PLY < yo} = P{X = xo} = 1 — F(x). (3.4-8) 


A repetition of the steps leading to (3.4-6) will again produce (3.4-6) except that 
the right side is negative. However, since the slope of T~'(y) is also negative, we 
conclude that for either type of monotonic transformation 


fd) =flT YN] To 


(3.4-9) 


or simply 


Sly) = Sel) 


dx 
rp (3.4-10) 


t Leibniz's rule, after the great German mathematician Gottfried Wilhelm von Leibniz (1646— 


1716), states that, if H(x, u) is continuous in x and u ang 


Biv) 
G(u) -| H(x, u) dx 
atu)” 


then the derivative of the integral with respect to the parameter w is 


dG(u) 
du 


= TILA), a4 Xu) = H{atw), u) “ - ‘e OH (x, om 
a pal 


uy 
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Exumple 3.4-1 If we lake T to be the linear transformation Y = T(Y) = 
aX + b, where a and b are any real constants, thea XY = 7" MY) = (Y - bya 
and dx/dy = 1/a. From (3.4-9) 


fla) “fl? a yat)| 


If X is assumed to be gaussian with the density function given by (2.4-1), 


a 


we get 
be — ous yuasentiest| 2 
fry) = el bia~ax}tf2ox? | _ . 
2na? a 
bs pam eer yn (eax + bYP/2a20x2 
2na7a2 


which is the density function of another gaussian random variable having 
dy =ady+b and of = aay 

Thus, a linear transformation of a gaussian random variable produces another 

gaussian random variable. A linear amplifier having a random voltage ¥ as its 

input is one example of a linear transformation. 


Nonmunotonic Transformations of a Continuous Random Variable 


A transformation may not be monotonic in the more general case, Figure 3.4-3 
illustrates one such transformation. There may now be more than one interval of 
values of X that correspond to the event {¥ < yo}. For the value of yy shown 
in the figure, the event {Y¥ < yo} corresponds to the event {X sx, and x) s 
X <x). Thus, the probability of the event {¥ < yo} now equals the probability 


Figure 3.4-3 A nonmonotonic 
transformation. (Adapted from 
Peebles (1976) with permission 
of publishers Addison-Wesley, 
Advanced Book Program) 
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of the event {x valucs yielding Ys Yo}, which we shall write as {x| ¥ S Yo}. In 
other words 


Fylyo) = P(Y S Yo} = P{x|Y < yo} = | Sixx) dx (3.4-11) 
tx1¥ < yo) 


Formally, one may differentiate to obtain the density function of Y: 


ee 1 | 
fro) = rm | fic(x) dx (3.4-12) 
Yo Jxl¥ syo) 


Although we shall not give a proof, the density function is also given by 
(Papoulis, 1965, p. 126) 


Sx(%n) 


dx 


x= Xn 


where the sum is taken so as to include all the roots x,, "= 1, 2,..., which are 
the real solutions of the equationt 


y= T(x) (3.4-14) 


We illustrate the above concepts by an example. 


Example 3.4-2, We find fy(y) for the square-law transformation 
Y¥ = T(X) = eX? 


shown in Figure 3.4-4, where c is a real constant c > 0. We shall use both the 
procedure leading to (3.4-12) and that leading to (3.4-13). 

In the former case, the event {Y < y} occurs when {-Jyle's xs 
Jyie} = {x|Y¥ < y}, 80 (3.4-12) becomes 


{ (YF. | 
Sry) = i [ves dx y20 


Upon use of Leibniz's rule we obtain 


— 
fi) = fal) asfve 14-7 vi 
iM yie) +S yO) ee 


aJey 


tify = TIX) has no real roots for a given value of y, then f,(y) = 0. 
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Figure 3.4-4 A square-law {rans- 
formation. (Adapted from Peebles 
(1976) with permission of publishers 
Addison-Wesley, Advanced Book 
x Program] 


In the latter case where we use (3.4-13), we have X= +./Y/c, Y 29, so 
x, = —-Jy/le and x, = /y/e. Furthermore, dT(x)/dx = 2cx so 


aT(X)| ex, = Ya oasley 
ie toss = 2cx, = 2 [P= —2/cy 
dT(x) 
Sh =2./ 

dx \emxy a 


From (3.4-13) we again have 


fil/yle) + Sl VV/0) 


Sly) = aJey 


y20 


Transformation of a Discrete Random Variable 


If X is a discrete random variable while Y = T(X) is a continuous transfor- 
mation, the problem is especially simple. Here 


Sel) = YL POX — Xn) (3.4-15) 


Fy(x) =X P(x dulx — Xp) (4-16) 


where the sum is taken to include all the possible valucs x,,0 = 1, 2,..., 08 X. 

If the transformation is monotonic, there is a one-to-one correspondence 
between X and Y so that a set {y,} corresponds to the set {x,} through the equa- 
tion y, = T(x,)- The probability P(y,) equals P(x,). Thus, 


Sed) = L Plynddly — Ya) (3.4-17) 


Fyy) =D Puy — Yo) (3.4-18) 


aac! 
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where 
Yn = T(X,) (3.4-19) 
P(y,) = P(x,) (3.4-20) 
If T is not monotonic, the above procedure remains valid except there now 


exists the possibility that more than one value x, corresponds to a valuc Ya In 


such a case P(y,) will equal the sum of the probabilities of the various x, for 
which y, = T(x,). : : 


PROBLEMS 


3-1 A discrete random variable X has possible values x, = i?, i= 1, 2, 3, 4, 5, 
which occur with probabilities 0.4, 0.25, 0.15, 0.1, and 0.1, respectively. Find the 
mean value X = E[X] of X. 
3-2 The natural numbers are the possible values of a random variable X; that is 
x, =n, n= 1, 2,.,... These numbers occur with probabilities P(x.) = (4). Fi 
the expected value of X. ‘i eae ee 
3-3 f the probabilities in Problem 3-2 are P(x,) =p", 0<p <1, show that 
ame 4, is the only value of p that is allowed for the problem as formulated. 
(Hint: Use the fact that f@,, fy(x) dx = | is necessary.) 
3-4 Give an example of a random variable where its mean value might not equal 
any of its possible values. 
3-5 Find: 

(a) the expected value, and 

(b) the variance of the random variable with the triangular density of Figure 
2.3-laifa = I/a. 
3-6 Show that the mean value and variance of the random variable having the 
uniform density function of (2,5-7) are: 


X = E[X] =(a + b)/2 
and 
a} = (b — a)*/12 
3-7 A pointer is spun on a fair wheel of chance numbered from 0 to 100 around 
its circumference, 
(a) What is the average value of all possible pointer positions? 
(b) What deviation from its average value will pointer position take on the 


average; that is, what is the pointer's root-mean-squared deviation from its 
mean? (Hint: Use results of Problem 3-6.) . 


3-8 Find: 
(a) the mean value, and 


(b) the variance of the random variable X defined by Problems 2-6 and 2-14 
of Chapter 2, 
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*3.9 For the binomial density of (2.5-1), show that 
E[X]) =X =Np 

and 
o% = Np! — p) 


3-10 (a) Let resistance be a random variable in Problem 2-11 of Chapter 2, Find 
the mean value of resistance. 

(b) What is the output, voltage E, if an average resistor were uscd in the 
circuit? : é 

(c) For the resistors specified, what is the mean value of E,?+Does the 
voltage of part (b) equal this value? Explain your results. 
3-11 (a) Use the symmetry of the density function given by (2.4-1) to justify that 
the parameter ay in the gaussian density is the mean valuc of the random vari- 
able: X = ay. 

(b) Prove that the parameter a% is the variance. (Hint; Use an equation from 
Appendix C.) 
3-12 Show that the mean value E[X] and variance o% of the Rayleigh random 
variable, with density given by (2.5-11), are 

BUX] =a + f/nb/4 
and 
o% = (4 — 2)/4 

3-13 What is the expected lifetime of the system defined in Problem 2-33 of 
Chapter 2? 
3-14 Find: 


(a) the mean value, and 
(b) the variance for a random variable with the Laplace density 


1 
ex — gu lx- mle 
Sx(x) 2b e 


where b and m are real constants, b > 0 and —0 <m< 0, 
3-15 Determine the mean value of the Cauchy random variable in Problem 2-34 
of Chapter 2. What can you say about the variance of this random variable? 
*3.16 For the Poisson random variable defined in (2.5-4) show that: 

(a) the mean value is b and 

(b) the variance also equals b. 
3-17 (a) Use (3.2-2) to find the first three moments My, Mz, and mty for the expo- 
nential density of Example 3.1-2. 

(b) Find m,, mz, and my; from the characteristic function found in Example 
3.3-1. Verify that they agree with those of part (a). 
3-18 Find expressions for all the moments about the origin and central moments 
for the uniform density of (2.5-7). 


a 


t 
i 
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3-19 Define a function g(-) of a random variable X by 
I xX 2 Xo 
X)= 
aX) ‘ x< Xo 
where Xo is a real number — 0 < Xo < 00, Show that 
Elg(X)] = 1 — Fx(%o) 


3-20 Show that the second moment of any random variable X about an arbi- 
trary point a is minimum when a=X; that is, show that E{(X — a)7] is 
minimum for a = X. 


4-21 For any discrete random variable X with values x, having probabilities of 


occurrence P(x,), show that the moments of X are 


N 
m= DL xiP(x) 


fmt 
N 
it, = x (x, — X)"P(x) 
=1 
where N may be infinite for some X. 


3-22 Prove that central moments H, are related to moments i, about the 
origin by 


HM, = y (i)-zrtm 
Keo 


4-23 A random variable X has a density function f,(x) and moments ™,. If the 


density is shifted higher in x by an amount a > 0 to a new origin, show that the . 


moments of the shifted density, denoted m,, are related to the moments m1, by 


m= > (tem 
k=O 


*3.24 Show that any characteristic function ,(w) satisfies 
|,(w)| S Ox(0) = 1 


3-25 A random variable X is uniformly distributed on the interval (—5, 15). 
Another random variable Y = e-X!3 is formed. Find EY]. 


3-26 A gaussian voltage random variable X [sce (2.4-1)] has a mean value X = 
ay = O and variance a} = 9. The voltage X is applied to a square-law, full-wave 
diode detector with a transfer characteristic Yi = 5X?. Find the mean value of the 
output voltage Y. 


*3.27 For the system having a lifetime specified in Problem 2-33 of Chapter 2, 
determine the expected lifetime of the system given that the system has survived 
20 wecks. 
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*3.28 The characteristic function for a gaussian random variable X, having a 
mean value of 0, is 


© fw) = exp (-0}.07/2) 


Find all the moments of X using ®y(@). 
*4.29 Work Problem 3-28 using the moment generating function 


Mx(v) = exp (6x 97/2) 


for the zero-mean gaussian random variable. 
*43.30 A discrete random variable X can have N + 1 values x, = kA, k =0, Loe 
N, where A > O is a real number. Its values occur with equal probability. Show 
that the characteristic function of X is 


1 sin ((N + !)wA/2] pinwsi2 
N+1 sin (wA/2) 
3-31 A random variable X is uniformly distributed on the interval (— 7/2, 2/2). X 
is transformed to the new random variable Y = T(X) = a tan (X), where a > 0. 
Find the probability density function of Y. 
3-32 Work Problem 3-31 if X is uniform on the interval (—7, 7). 
3-33 A random variable X undergoes the transformation Y = a/X, where a is a 
real number. Find the density function of Y. 
4-34 A random variable X is uniformly Uistributed on the interval (—a, a). It is 
transformed to a new variable Y by the transformation Y = cX? defined in 
Example 3.4-2. Find and sketch the density function of Y. 
3-35 A zero-mean gaussian random variable X is transformed to the random 
variable -Y determined by 


®,(w) = 


cm X>0 
0 X <0 


where c is a real constant, c > 0, Find and sketch the density function of Y. 

3-36 If the transformation of Problem 3-35 is applied to a Rayleigh random vari- 
able with a 2 0, what is its effect? 

*3.37 A random variable © is uniformly distributed on the interval (8,, 82) where 
0, and 0, are real and satisfy 


0<0,<0,<% 


Find and sketch the probability density function of the transformed random vari- 
able Y = cos (©). 
3-38 A random variable X can have values —4, —1, 2, 3, and 4, each with prob- 
ability '/s. Find: 

(a) the density function, 

(b) the mean, and 

(c) the variance of the random variable Y = 3X3. 


: * 


ra 
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ADDITIONAL PROBLEMS 


3-39 (a) Find the average amount the gambler in Problem 2-42 can expect to 
win. (6) What is his probability of winning on any given playing of the game? 
3-40 The arcsine probability density is defined by 

rect (x/2a) 

n/a? — x? 

for any real constant a > 0. Show that ¥ =Oand X? = a*/2 for this density, 


*3-41 For the animal described in Problem 2-58 find its expected lifetime given 
that it will not live beyond 20 weeks. i 


3-42 Find the expected value of the function g(X) = X° where X is a random 
varia ble defined by the density : 


Iy(x) = 


Selx) = (7h)u(x) exp (—x/2) 
3-43 Continue.Problem 3-25 by finding all moments of Y. (Hint: Treat Y" as a 
function of Y, not as a transformation.) 
3-44 Reconsider the production line that manufactures bolts in Problem 2-12. 
(a) What is the average length of bolts that are placed up for sale? 
(6) What is the standard deviation of length of bolts sold? 


(c) What percentage of all bolts sold are expected to have a length within 
One standard deviation of the average length? 


(d) By what tolerance (as a percentage) does the average length of bolts sold 
match the nominally desired length.of 760 mm? 


3-45 A random variable X has a probability density 
fihe (1/16) cos (nx/8) —~4<x<s4 
ee 0 elsewhere 
Find: (a) its mean value X, (b) its second moment X?, and (c) its variance. 


3-46 A certain meter is designed to measure small de voltages but makes errors 
because of noise. The errors are accurately represented as a gaussian random 
variable with a mean of zero and a standard deviation of 107? V. When the dc 
voltage is disconnected it is found that the probability is 0.5 that the meter 
reading is positive due to noise. With the dc voltage present this probability 
becomes 0.2514. What is the de voltage? 


3-47 Find the skew and coefficient of skewness for a Rayleigh random variable 
for which a = 0 in (2.5-11), 


3-48 A random variable X has the density 
_ SChaX—x? + 8x—12) 25x56 
fils) = i elsewhere 


Find the following moments: (a) mg, (b) m,, (€) mz, and (d) fy. 
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3-49 The chi-square density with N degrees of freedom is defined by 
iNi2y- 4 


Sx(x) = PAN) 


where I'(-) is the gamma function 


u(xje 7 */? 


T(z) = [ome dé real part ofz>0 
{¢) 


and N = 1, 2,.... Show that (a) X = N, (8) X? = N(N + 2), and (c) 0% = 2N for 
‘this density. . Ge 
3-50 For the density of Problem 3-49 find its arbitrary moment x", n= 
0,1,2,.... . 
3-51 A random variable X is called Weibullt if its density has the form 

fx(x) = abx?~! exp (—ax?)u(x) 


where a>0 and b>O are real constants. Use the definition of the gamma 
function of Problem 3-49 to find (a) the mean value, (b) the second moment, 


"+ and (c) the variance of X. 


*4.52 Show that the characteristic function of a random variable having, the bino- 
mial density of (2.5-1) is 
Dy(w) = [1 — p + pe)" 
*3.53 Show that the characteristic function of a Poisson random variable defined 
by (2.5-4) is 
D(w) = exp [—b(1 — e/*)] 
*3.54 The Erlangt random variable X has a characteristic function 


a |" 
oo | 75 


for a>0 and N=1, 2, .... Show that X = N/a, X? = N(N + 1)/a?, and 
a2 = N/a’. : = ; . 

3-55 A random variable X has X = a hiek: = 11, hs oy =2. For a new 
random variable Y = 2X — 3, find (a) ¥,(b) Y*, and (c) oy. , ; 
*3.56 For any real random variable X with mean X and variance oy, Chebychev's 

Inequality§ is 


P{|X —X| = doy} < 1/? 


i i i int: Define a new random 
* where 4 > O is a real constant, Prove the inequality. (Hint: De 
variable Y = 0 for |X —X|<Aoy and ¥ = A?o} for [X — X| > Aox, observe 
that Y < (X — X)? and find ELY).) 


+ After Ernst Hjalmar Waloddi Weibull (1887- - ), a Swedish upplied physicist. 
i - : ish engineer, 

JA. K. Erlang (1878-1929) was a Danis f 

§ After the Russian mathematician Pafnuty Lvovich Chebychev (1821-1894). 
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3-57 A gaussian random variable, for which 


fal) = (2//') exp (— 437) 
is applied to a square-law device to produce a new (output) random variable 
Y = X?/2. (a) Find the density of Y. (b) Find the moments m, = FLY"), 2 = 
0, 1, .... (Hint: Put your answer in terms of the gamma function defined in 
Problem 3-49.) 
3-58 A gaussian random variable, for which X = 0.6 and oy = 0.8, is trans- 
formed to a new random variable by the transformation 


4 10<X <0 

oe 2 0<sX <10 

me Te -10< xX <0 
-4 ~oa<X<-1.0 


(a) Find the density function of Y. 

(b) Find the mean and variance of Y. 
3-59 Work Problem 3-31 except assume a transformation Y = T(X) =a sin (X) 
with a > 0. 
3-60 Let X be a gaussian random variable with density given by (2.4-1). If X is 
transformed to a new random variable Y = b + e*, where b is a real constant, 
show that the density of Y is log-normal as defined in Problem 2-35. This trans- 
formation allows log-normal random numbers to be generated from gaussian 
random numbers by a digital computer. 
3-61 A random variable X is uniformly distributed on (0, 6). If X is transformed 
to a new random variable Y = 2(X — 3)? — 4, find: (a) the density of Y, (b) y, 
(c) 3. 


CHAPTER 


FOUR 
MULTIPLE RANDOM VARIABLES 


4.0 INTRODUCTION 


In Chapters 2 and 3, various aspects of the theory of a single random variable 
were studied. The random variable was found to be a powerful concept. It 
enabled many realistic problems to be described in a probabilistic way such that 
practical measures could be applied to the problem even though it was random. 
For example, we have seen that shell impact position along the line of fire from a 
cannon to a target can be described by a random variable (Problem 2-29). From 
knowledge of the probability distribution or density function of impact position, 
we can solve for such practical measures As the mean value of impact position, its 
variance, and skew. These measures are nol, however, a complete enough descrip- 
tion of the problem in most cases. 

Naturally, we may also be interested in how much the impact positions 
deviate from the line of fire in, say, the perpendicular (cross-fire) direction. “In 
other words, we prefer to describe impact position as a point in a plane as 
opposed to being a point along a line. To handle such situations it is necessary 
that we extend our theory to include two random variables, one for each coordi- 
nate axis of the plane in our example. In other problems it may be necessary to 
extend the theory to include several random variables. We accomplish these 
extensions in this and the next chapter. 

Fortunately, many situations of interest in engineering can be handled by the 
theory of two random variables.t Because of this fact, we emphasize the two- 
variable case, although the more general theory is also stated in most discussions 
to follow. oe 2 


t In particular, it will be found in Chapter 6 that such important concepts as autocorrelation, 
cross-correlation, and covariance functions, which apply to random processes, are based on two 
random variables. 
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4.1 VECTOR RANDOM VARIABLES 


Suppose two random variables X and Y are defined on a sample space S, where 
sp¢cific values of X and Y are denoted by x and y, respectively. Then any ordered 
pair of numbers (x, y) may be conveniently considered to be a random point in the 
xy plane. The point may be taken as a specific value of a vector random variable 


or a random vector. Figure 4.1-1 illustrates the mapping involved in going from . : 


S to the xy plane. 

The plane of all points (x, y) in the ranges of X and Y may be considered a 
new sample space. It is in reality a vector space where the components of any 
vector are the values of the random variables X and Y. The new space has been 
called the range sample space (Davenport, 1970) or the two-dimensional product 
space. We shall just call it a joint sample space and give it the.symbol S,. 

As in the case of one random variable, let us define an event A by 


A={X <x} (4.1-L) 
A similar event B can be defined fory: 
B={Y sy} (4.1-2) 


Events A and B refer to the sample space S, while events {X sx}jand{Y¥< 


refer to the joint sample space S,.t Figure 4.1-2 illustrates the correspondences 


ft There are some specific conditions that must be satisfied in a complete definition of a random 
vector (Davenport, 1970, Chapter 5). They are somewhat advanced for our scope and we shall simply 
assume the validity of our random vectors. 

{Do not forget that elements s of S form the link between the two events since by writing (X < x} 


we really refer to the set of those s such that X(s) < x for some real number x. A similar statement 
holds for the event {Y < y}. 


Sy 
moma (X tn), 150) 


Function X 


Figure 41-1 Mupping from the sample spuce S to the joint sample space S, (xy plane). 
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Figure 4.f-2, Comparisons of events in S with those in S,, 


between events in the two spaces. Event A corresponds to all points in S, for 
which the X coordinate values are not greater than x. Similarly, event B corre- 
sponds {o the Y coordinate values in S, not exceeding y. Of special interest is 
to observe that the event A a B defined on S corresponds to the joint event 
{X <x and Y < y} defined on S,, which we write {X <x, ¥ sy}. This joint 
event is shown crosshatched in Figure 4.1-2. 

In the more general case where N random variables X,, X2,.... Xy are 
defined on a sample space S, we consider them to be components of an N- 
dimensional random vector or N-dimensional random variable. The joint sample 
space S, is now N-dimensional. 


4.2 JOINT DISTRIBUTION AND ITS PROPERTIES 


The probabilities of the two events A = {X <x} and B=({Y < y} have already 
been defined as functions of x and y, respectively, called probability distribution 
functions: 


F x(x) = P(X <x} (4,2-1) 
Fyy) = PLY < y} (4,2-2) 


We must introduce a new concept to include the probability of (he joint event 
{X <x, Y sy}. 


Joint Distribution Function 


We define the probability of the joint event {X¥ < x, Y < y}, which is a function 
of the numbers x and y, by a joint probability distribution function and denote it 
by the symbol Fy y(x, y). Hence, 


Fy A(x, y) = PIN sx, ¥ sy} (4.2-3) 
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It should be clear that P{X <x, Y <y} = P(A B), where the joint event 
Ac Bis defined on S. 


To illustrate joint distribution, we take an example where both random varis 


ables X and Y are discrete. 


I 


Example 4.2-1 Assume that the joint sample space S, has only three possible 
elements: (1, 1), (2, 1), and (3, 3). The probabilities of these clements are 
assumed to be P(i, 1) = 0.2, P(2, 1) = 0.3, and P(3, 3)=0.5. We find 
Fy, r(x, y). 

In constructing the joint distribution function, we observe that the event 
{X <x, Y sy} has no elements for any x <1 and/or y <1. Only at the 
point (1, 1) does the function assume a step valuc. So long as x2 1 and 
yo 1, this probability is maintained so that Fy, y(x, y) has a stair step 
holding in the region x 2 land y2 1 as shown in Figure 4.2-la. For larger x 
and y, the point (2, 1) produces a second stair step of amplitude 0.3 which 
holds in the region x 22 and y 21. The second step adds do the first. 
Finally, a third stair step of amplitude 0.5 is added to the first two when x 
and y are in the region x 2 3 and y 23. The final function is shown in 
Figure 4.2-La, 


pe ev ee ee ee 


The preceding example can be used to identify the form of the joint distribu- 
tion function for two general discrete random variables. Let X have N possible 
values x, and Y have M possible valucs y,,, then 


NM 
Fy, r(% y) = ry 2 Pn aes < x,)u(y a 1) (4.2-4) 
where P(Xq1 Ym) is the probability of the joint event {X =x,, Y = Ym} and u(+) is 
the unit-step function. As seen in Example 4.2-1, some couples (Xn, Ym) May have 
zero probability. In some cases N or M, or both, may be infinite. 
If Fy, p(x, y) 1s plotted for continuous random variables X and Y, the same 
general behavior as shown in Figure 4.2-1a is obtained except the surface 
becomes smooth and has no stairstep discontinuities. 
For N random variables X,, n= 1, 2, ++ N, the generalization of (4.2-3) is 
direct. The joint distribution function, denoted by Fy xacn XM by X20 00 Xy)s is 
defined as the probability of the joint event {X, SX X2S Xa Xy S Xn}! 


Fx. vaca kaXts X20 eeey Xn) = P{X, SX, X21 S200 XySxXw} (4.2-5) 


For a single random variable X, we found in Chapter 2 that F(x) could be 
expressed in general as the sum of a function of stairstep form (due to the discrete 
portion of a mixed random variable X) and a function that was continuous (due 
to the continuous portion of X). Such a simple decomposition of the joint dis- 
tribution when N > [ is not gencrally truc [Cramér, 1946, Section 8.4]. However, 
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Figure 4.2-1 A joint distribution function 
(a), and its corresponding joint density 
function (b), that apply to Examples 
4.2-1 and 4,2-2. 


it is true that joint density functions in practice often correspond to all random 


variables being either discrete or continuous. Therefore, we shall limit our con- 


sideration in this book almost entirely to these two cases when N > 1. 


Properties of the Joint Distribution 


A joint distribution function for two random variables X and Y has several 
properties that follow readily from its definition. We list them: 


(l) Frxl-o, —)=0 Fe (-o, a0 Fare oo) = 0 42-60) 
(2) Fx, (00, 20) = 1 (4.2-6b) 
(3) OS Fx x,y) S1 (4.2-6c) 
(4) Fx, r(x, y) is a nondecreasing function of both x and y (4.2-6d) 
(5) Fy, v2» Ya) + Fx, v1» y1) — Fx, 1 Ya) — Fx, vas yi) 

= P(x, <XSx2,< VS y2} 20 (4.2-6e) 
(6) Fy, yl, oo) = F(x) Fy, (0, y) = Fy) (4.2-6f) 
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. functions Fy(x) or Fy(y) obtained in this manner are called marginal distribution 
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The first five of these properties are just the two-dimensional extensions of a ‘S 


the properties of one random variable given in (2.2-2). Properties 1, 2, and Smay 


be used as tests to determine whether some function can be a valid distribution ° z 4 
function for two random variables Y and Y (Papoulis, 1965, p. 169). Property 6 me 3 


deserves a few special comments. 


Marginal Distribution Functions \ 


Property 6 above states that the distribution function of one random variable can 
be obtained by setting the value of the other variable to infinity in Fy y(x, y). The 


Sunctions, 


To justify property 6, it is easiest to return to the basic events A and B, de- - 
fined by A = {X < x} and B={Y < y}, and observe that Fy y(x, y) = P{X <x, 
Y s y} = P(A 2 B). Now if we set y to ©, this is equivalent to making B the cer- > 
tain event; that is, B= {Y sao} =S, Furthermore, since AN B= ANS = A, *¢ 
then we have Fy ,(x, 0) = P(A A S)= P(A) = P{X s x} = Fy(x). A. similar 
proof can be stated for obtaining F',(y). 


eS 


Example 4.2-2 We find explicit expressions for Fy, (x, y), and the marginal 
distributions Fy(x) and Fy(y) for the joint sample space of Example 4.2-1. 


The joint distribution derives from (4.2-4) if we recognize that only three 
probabilities are nonzero: 


Fy, vx, y) = PL, Dux — tuy — 1) 
+ P(2, I)u(x — 2)u(y — 1) 
+ P(3, 3)u(x — 3)u(y — 3) 
where P(I, 1) = 0.2, P(2, 1) = 0.3, and P(3, 3) = 0.5. If we set y = co: 
F(x). = Fy, p(x, 0)... 
= PCL, Iulx ~ 1) -+ P(2, Wu(x — 2) + P(3, 3)u(x — 3) 
= 0.2u(x — 1) + 0.3u(x — 2) + 0.5u(x — 3) 
If we sel x = 00; 
Fy) = Fx, y(o, y) 
= 0.2u(y — 1) + 0.3u(y — 1) + 0.5u(y — 3) 
= 0.5u(y — 1) + 0.Su(y — 3) 


Plots of these marginal distributions are shown in Figure 4.2-2, 


OO OO 
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Fy (x) 
1.0]- 


F, 
as 1.0 
1.0 
0.5 
Figure 4.2-2 Marginal distributions applicable 
0 ! 2 3 Y to Figure 4.2-t und Example 4.2-2: (a) Fy(x) 


() and (b) Fy{y). 


From an N-dimensional joint distribution function we may obtain a k- 
dimensional marginal distribution function, for any selected group of k of the N 
random variables, by setting the values of the other N — k random variables to 
infinity. Here k can be any integer 1, 2, 3,...,N — 1. 


4.3 JOINT DENSITY AND ITS PROPERTIES 


In this section the concept of a probability density function is extended to include 
multiple random variables. 


Joint Density Function 


For (wo random variables X and Y, the joint probability density function, denoted 
Sx. y(x, y), is defined by the second derivative of the joint distribution function 
wherever it exists: 


2 
fu, 9) = AR (4.31) 


We shall refer often to fy, y(x, y) as the joint density function. 
If X and Y are discrete random variables, Fy. y(x, y) will possess step discon- 
tinuities (see Example 4.2-! and Figure 4.2-1). Derivatives at these discontinuities 


“er 
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Properties 1 and 2 may be used as sufficient tests to determine if some function 

_ can be a valid density function. Both tests must be satisficd (Papoulis, 1965, 
p. 169). 

The first five of these propertics arc readily verified from earlier work and the 

‘reader should go through the necessary logic as an exercise. Property 6 intro- 


are normally undefined. However, by admitting impulse functions (sce Appendix 
A), we are able to define fy, r(x y) at these points. Therefore, the joint density Bi) 
function may be found for any two discrete random variables by substitution of MBER 


(4.2-4) into (4.3-1): 


N M 
fers N= DL LP Ym) (x = Xn) SY = Yn) (4.3-2) duces a new concept. 
. n=imel 


An example of the joint density function of two discrete random variables is 


shown in Figure 4.2-1b. 
When N random variables X14, Nas ees Xv» are involved, the joint density 
function becomes the N-fold partial derivative of the N-dimensional distribution 


function: 


Marginal Density Functions 


The functions fy(x) and f(y) of property 6 arc called marginal probability density 
functions or just marginal density functions. They are the density functions of the 


ONF yy xy conn Hy Fas re Xn) (4.3-3) single variables X and Y and are defined as the derivatives of the marginal dis- 


. ARIE Se. 2 rerere 5. 1s0l Leet SE 
Sara Kaen ty Xa oo Xn) Ox, OX, °°' OXy tribution functions: 
By direct integration this result is equivalent to AF yx 
Sx) = a . (4.3-6) 
Fyy. xaccuXnOX Nay cees Nw) x 
xN x2 xt dF y(y) 
= { _ { { Sorc Xan talSie Sayer Ey) dé, do. 0* dew (4.3-4) Sb) = dy (43-7) 


By suvstituting (4.3-5d) and (4.3-5e) into (4.3-6) and (4.3-7), respectively, we are 
able to verify the equations of property 6. 
We shall illustrate the calculation of marginal density functions from a given 


joint density function with an example. 


Properties of the Joint Density 


Several properties of a joint density function may be listed that derive from its 
definition (4.3-1) and the properties (4.2-6) of the joint distribution function: 


(1) fxr 9) 2 0 (4.3-5a) 
i Cee iia ® i 22 Example 4.3-1 We find f,(x) and fj (y) when the joint density function is given’ 
(2) fe [7 fasts y) dx dy = 4 (4.3-5h) by (Clarke and Disney, 1970, p. 108): 
y x = —x(yt 1) 
(3) Fy. ux, y) = { | fe. aes &,) dé, dé, (4.3-5¢) Sx, y(%, y) u(x)u(y)xe 
cuca From (4.3-59) and the above equation: 
4) Fy(x) = (Ey, 2) dba dé (4.3-5d) : © ne © 
ae) i [ier wae ace fds) = \ u(x)xe 204) dy = u(x)xe~* | e~*? dy 
. my y 2 Bx: \ 0 fv) 
Fy) = | | Seu &) dg, ae, (4.3-5e) iq = u(x)xe7 *(1/x) = u(xje™* 
yn 3 after using an integral from Appendix C. 
(5) P(x, <X Stn <¥ Sv) = { fares yydx dy (43-5/) From (4.3-5h): 
yt x 
6) fpr] | Serbs 9) dy (4.3-54) Jyya [aye 208? dee OP, 
(6) Sx(x) = ne x r% Yd) r i (y + 1)? 


is after using another integral from Appendix C. 
(y) = | Su. r% y) dx (4.3-5h) 
fi ) seg: x.Y 
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For N random variables X,, X2,..., Xy, the k-dimensional marginal density § 
function is defined as the k-fold partial derivative of the k-dimensional marginal ° 
distribution function. It can also be found from the joint density function by inte- 


grating out all variables except the k variables of interest X,,.X2,..., X4: 


Pa Naveed X05 coy Xe) 


wo 


=| | Sey. xa, re xnX i Xan cee Xy) Xa UXyga te dxy  (4.3-8) : hii 


4.4 CONDITIONAL DISTRIBUTION AND DENSITY 
\ 


In Section 2.6, the conditional distribution function of a random variable X, b 


given some event B, was defined as 


P{X <x mB) 


Fx(x|B) = P(X <x]B} = PB) 


for any event B with nonzero probability. The corresponding conditional density P : 


function was defined through the derivative 


dF ,(x| B) 


"Sale| B) = 


(4.4-2) 


In this section these two functions are extended to include a second random vari- 


able through suitable definitions of event B. 


Conditional Distribution and Density—Point Conditioning 


Often in practical problems we are interested in the distribution function of one 
random variable X conditioned by the fact that a second random variable Y has 


some specific value y. This is called point conditioning and we can handle such * 


problems by defining event B by 
B={y-—Ady<Ysy+ Ay} (4.4-3) 


where Ay is a small quantity that we eventually let approach 0. For this event, 
(4.4-1) can be written 


rat feo Sx, vlEr €2) dé, db, 
Fy(xly — dy < ¥ sy + Ay) = eee ee 
: eee acs BER flO de 
where we have used (4.3-5f) and (2.3-6d). 
Consider two cases of (4.4-4). In the first case, assume X and Y are both dis- 
crete random variables with values x,, i= 1, 2,..., N, and y,, f/=1,2,..., M, 
respectively, while the probabililies of these values are denoted P(x;) and P(y,), 


441) 2 


(44-4) : 
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respectively. The probability of the joint occurrence of x, andy, is denoted 
P(x,, y,). Thus, 


M 
Sly) = XPV) 8b - yD) tetil 
jut 
NM , 6 
Serbs y= LL Pony) dx — x) 6 ~ yp) aoe 
i=l jet 


g Now suppose that the specific value of y of interest is Yas With substitution of 
- (4.4-5) and (4.4-6) into (4.4-4) and allowing Ay— 0, we obtain 


° 


. P(x, , Yn) 
ive Fae fs kh (4.4-7) 
F,(x| ¥ Ys) py P(y,) ( i 
: After differentiation we have 
x P(x, yx) 
Yey)=a Yo ix — x) (4.4-8) 
Sx(x| Yr) p> P(y,) ( 4 


re 
Example 4.4-1 To illustrate the use of (4.4-8) assume a joint censily function 
as given in Figure 4.4-la. Here P(x, yi) = 45, Plx2s i) = Ys, ele. Since 
Ply) = is) + hs) = is» use of (4.4-8) will give fy(x| ¥ = ys) as shown in 
Figure 4.4-1b. 

Sa nn nner 


The second case of (4.4-4) that is of interest corresponds to X and Y both 
continuous random variables, As Ay— 0 the denominator in (4.4-4) becomes 0. 
However, we can still show that the conditional density fy(x| Y = y) may exist. If 
Ay is very small, (4.4-4) can be written as 


Xo Sx r(Si» y) doi 24) (4.4-9) 


Fy(xly- Ay <¥ syt dy)= Srly)2ay 


and, in the limit as Ay— 0 


ie i 
Fyx|Y = y)= =% a _ (4.4-10) 


for every y such that fy(y) 4 0. After differentiation of both sides of (4.4-10) with 
respect 0 Xx: 


Se y(™ y) 
= ye (4.4-11) 
Sy(x| Y y) fry) 
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Figure 4.4-4 A joint density function 
(a) and a conditional density function 
(b) applicable to Example 4.4-1. 


When there is no confusion as to meaning, we shall often write (4.4-11) as 


fybely) = (44-12) 


It can also be shown that 


ee Sx, y(%_¥) 
Sey |x) = Tule) (4.4-13) 


. 


Example 4.4-2 We find fry |x) for the density functions defined in Example 
4,3-1, Since 

feos) = u(x)u(y)xe 7207) 
and 


Sxl) = uxje™ 
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are nonzero only for 0 < y and 0 < x, yy 1x) is nonzero only for 0 < y and 
O<x.Itis 


fr lx) = u(x)u(y)xe 7“ 


from (4.4-13). 


«Conditional Distribution and Density—Interval Conditioning 
It is sometimes convenient to define event B in (4.4-1) and (4.4-2) in terms ofa 
random variable Y by 

B={y,<YS yp} (4.4-14) 
where y, and y, are real numbers and we assume P(B) = Pty, < Ys ye} 0. 
With this definition it is readily shown that (4.4-1) and (4.4-2) become 
Fy lx. Ya) = By y(Xs Ja) 

Fy(ys) — F(a) 


= mit SxS y) dg dy ics 
eer Sx. vi% y) dx dy (4.4-15) 


Fy(xlya< Y SY) = 


and 


_ _f hs Sx. ¥% y) dy 
Slxl Ya < Y SY») Frye Seals, 9) dx dy (4.4-16) 


These last two expressions hold for X and Y either continuous or discrete 
random variables. In the discrete case, the joint density is given by (4.3-2). The 
resulting distribution and density will be defined, however, only for y, and yy 
such that the denominators of (4.4-15) and (4.4-16) are nonzcro. This requirement 
is satisfied so long as the interval y, < y Sy» spans at least one possible value of 
Y having a nonzero probability of occurrence. 

An example will serve to illustrate the application of (4.4-16) when X and Y 
are continuous random variables. 


Se ee 
Example 4.4-3 We use (4.4-16) to find fx Y s y) for the joint density func- 
tion of Example 4.3-1. Since we have here defined B= {Y S y}, then y, = 
—oo and y, = y. Furthermore, since fy, :(x, y) is nonzero only for 0 < x and 
0 < y, we need only consider this region of x and y in finding the conditional 
density function. The denominator of (4.4-16) can be written asf”. Sy(4) dé, 
By using results from Example 4.3-1; 


: Oy ae ieee pe ee 
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Figure 4.4-2 Conditional probability density functions applicable to Example 4.4-3, 


and zero for y < 0, after using an integral fi i 
Sle eae g£ gral from Appendix C. The numerator 


| Sx, vx, €) dg = [[eoreae dé 
ies : 


y 
u(x)xe™* | en de 


0 

= u(x)e~*(1 — e7*”) y>0o0 
and zero for y < 0, after using another integral from Appendix C, Thus 
ev —e *?) 


This function is plotted in Figure 4.4-2 for several values of y. 


y+ 


Sx|Y s y= wor 


4.5 STATISTICAL INDEPENDENCE 


Tt will be recalled from (1.5-3) that two events A and B¢ alistically i 
dent if (and only if) : nd B are statistically indepen-, 


P(A 1 B) = P(A)P(B) (4.5-1) 
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This condition can be used to apply to two random variables ¥ and Y by delin- 
jing the events A = {X < x} and B= {Ys y} for two rea} numbers x and sy. 
Thus, X and Y are said to be statistically independent random variables if (and 


only if} 
P{X <x, ¥ sy} = P(X sx}P{Y sy} (4.5-2) 
From this expression and the definitions of distribution functions, it follows 
that 
Fy, p(x, y) = FxQ)F yO) _ (45-3) 


if X and Y are independent. From the definitions of density functions, (4.5-3) 
gives 

fe Y) = LOOLO) (4.5-4) 
by differentiation, if X and Y are independent. Either (4.5-3) or (4.5-4) may serve 


ag a suMicient definition of, or test for, independence of two random variables, 
The form of the conditional distribution function for independent events is 


* found by use of (4.4-1) with B = {Y sy}: 


P{X <x, Y sy} Fy r(x, Y) 
FAUxlY < y= ooo 4,5-5) 
RES ey 
By substituting (4.5-3) into (4.5-5), we have 
F(x] ¥ sy) = Fx) (4.5-6) 


In other words, the conditional distribution ceases to be conditional and simply 
equals the marginal distribution for independent random variables. I{ can also be 
shown that _ 


Fy(y|X <x) = Frly) (4.5-7) 


Conditional density function forms, for independent X and Y, are found by 
differentiation of (4.5-6) and (4.5-7): 


SlxlY sy) = fx(x) (4.5-8) 
LOX sx) =frly) (4.5-9) 


I 
Example 4.5-1 For the densities of Example 4.3-1: 


~xlyt ly 


Su, Qs y) = u(x)uly)xe 


Sed Sey) = HQULy) ras #S x. 9) 


Therefore the random variables X and Y are not independent. 
Se ee ee ———————— ee 
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In the more general study of the statistical independence of N random vari- 
ables Ny, Xqacce, Kye We define events A; by 


Apa{Xisxho PRN N (4.5-10) 


where the x, are real numbers. With these definitions, the random variables X, 
are said to be statistically independent if (1.5-6) is satisfied. 

It ean be shown that if X 1, Xasees Xy are statistically independent then any 
group of these random variables is independent of any other group. Furthermore, 
a function of any group is independent of any function of any other group of the 
random variables. For example, with N = 4 random variables: X4 is independent 
of X,+X,+ X13 Xs is independent of X,+ Xj, ete. (sce Papoulis, 1965, 
p. 238). 


4.6 DISTRIBUTION AND DENSITY OF 
A SUM OF RANDOM VARIABLES 


The problem of finding the distribution and density functions for a sum of sta- 
tistically independent random variables is considered in this section. 


Sum of Two Random Variables 


Let W’ be a random variable equal to the sum of \wo[independent|andom vari- 


ables X and Y: 
W=X+Y (4.6-1) 


This is a very practical problem because X might represent a random signal 

voltage and Y could represent random noise at some instant in time. The sum W 

would represent a signal-plus-noise voltage available to some receiver. 
The probability distribution function we seck is defined by 


Fy(w) = P{W sw} = P(X +Y sw} (4.6-2) 


Figure 4.6-1 illustrates the region in the xy plane where x + y SW. Now from 
(4.3-5f), the probability corresponding to an elemental area dx dy in the xy plane 
located at the point (x, J) is fy, A(X y) dx dy. If we sum all such probabilities over 


the region where x + y S wwe will obtain Fyy(w). Thus 


Co) wry 
Fy(w) = [: | Sy, 0% y) dx dy (4:6-3) 
and, after using (4.5-4): 
2 wry 
Fy(w) = | Sly) | Selx) dx dy (4.6-4) 
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Figure 4.6-1 Region in xy plane where x + 
ys 


By differentiating (4.6-4), using Leibniz’s rule, we get the desired density function 
oO 
Siw) = { LS — y) dy (4.6-5) 
-~© 
This expression is recognized as a convolution integral. Consequently, we have 


shown that the density function of the sum of two statistically independent random 
variables is the convolution of their individual density functions. 


oa ee ea 
Example 4.6-1 We use (4,6-5) to find the density of WHxX+Y where the 
densities of X and Y arc assumed to be 


Iolo) = + Cus) — ul = a) 


Ju) = 5 Lay) — wy — A 


with 0 <a <b, as shown in Figure 4.6-2a and b. Now because 0 < X and 
0 < Y, we only need examine the case We X+Y> 0. From (4.6-5) we 
wrile 

Siv(w) = | " [u(y) — u(y — b)JLu(w — y) — uw ~ y ~a)] dy 


{ oO 
ers | [1 — wy — by] [uw — y) — uw — y — a)] dy 


i oe « 
-4 | uw- 9 ay-[ u(w — y — a) dy 


- [wy — byu(w — y) dy + { 
o 0 


a 


u(y Zs b)u(w — y — a) iy| 
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s** because X, is independent of both X, and X,. Thus, by applying (4.6-5) to the 
two variables X, and Y, to find the density function of Y; = X3 + Y,, we get 


Si yaxy+xa4xsl¥2) = fx,(3) * fy exe xis) 
= fy (x3) * Se,(x2) * fr, 0 (4.6-6) 


By continuing the process we find that the densit function of Y=X, +4, + 


Ais ses + Xy is the (N_— 1)-fold convolution of the N individual density functions: 
LA) = SalXn) * fey (Xn—1) wore a fy (X4) -  (4.6-7) 


The distribution function of Y is found from the integral of fy(y) using 
(2.3-6c). 


Se h%2) * fe Q01)-t Next, we know that X4 will be independent of Y, = X, + Ny 


(a) 
Sly) 


Ll bad 


rare | 7 [ ey 


*4.7 CENTRAL LIMIT THEOREM | 


Broadly defined, the central limit theorem says that the robability distribution 
function of the sum ofa large number of random variables approaches a gaussian *s 
distribution. Although the theorem is known to apply to some cases of sta- 
listically dependent random variables (Cramér, 1946, p. 219), most applications, me 
and the largest body of knowledge, are directed toward statistically independent 
random variables. Thus, in all succeeding discussions we_ assume statistically 


independent random variables. J 


0 a 6 a+b 


“Figure 4.6-2 Two density functions (a) 


a (c) and (6) and their convolution (c). * Unequal. Distributions 


Let X, and a} 


be the means_and_v. 


ariances, respectively, of N random variables | 
densities, The central 


All these integrands are unity; the values of the integrals are determined by 


; the unit-step functions through their control over limits of integration. After imit theorem states that the sum Yy = Ay 42 70 ay wee ee Ye Xj te Ret Ay wily Nis mend 
i straightforward evaluation we get ‘ Wa X +X, +--+ Xy und variance oy, = 0%, +03, 4" + Fry, has _a a 
" probability distribution that asymptotically a i paussian as N — 0. i 
ae w/ab . Osw<a . ecessary conditions for the theorem’s validity are difficult to state, but sufficient : 
Sul) = 1/b asw<b conditions are known lo be (Cramér, 1946; Thomas, 1969) ry 
Ww ‘7, 4 
é la bsw<atb ah >B,>0 f= 1,.,N (4.7-1a) a 
AK weoath 7 
| ee huis ELL X,-¥,P]<B, f=1,2,...N (4.7-1b) 
‘3: which is sketched in Fi - it 543 
3 n Figure 4.6-2c. where B, and B, are positive numbers. These conditions guarantee thal no one . 
{ee fandom variable in the sum dominates. . 
Hered The reader should observe that the central limit theorem guarantees only 
‘ 3 *Sum of Several Random Variables that the distribution of the sum of random variables becomes gaussian, It docs 1) 
F : hot follow that the probability density is always gaussian. For continuous 4 
t 3 When the sum Y of N independent random variables X,, X,,..., Xy is to Cd ee P y y3 ys_Bi 4 
} ‘ be considered, we_may extend the above analysis for two random variables, 
aie Let ¥, =X, +X,. Then we k f } i i i 
i 1 1 2 e know from the preceding work that /,,(y,) = ¢ The asterisk denotes convolution. cc 
; Eas 


a 


106 PRODANILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 


random variables there is usually no problem, but certain conditions imposed on 
the individual random variables (Cramér, 1946; Papoulis, 1965 and 1984) will 
guarantec that the density is gaussian. 
For discrete random variables X the sum Yy will also be discrete so its 
density will contain impulses and is, therefore, not gaussian, even though the dis- 
tribution approaches i 
Variable are kb, koe OQ) ob 1, 2, 0... with ba constant tthe envelope of the 
mpulses in the density of the sum will be gaussian (with mean ¥, and variance 
a2,). This case is discussed in some detail by Papoulis (1965). 
The practical uscfulness of the central limit theorem docs not reside so much 
in the exactness of the gaussian distribution tor N— oo because the variance of 
Yy becomes infinite from (4.7-1a). Usefulness derives more from the fact that Yy 


for finite N may have a distribution that is closely approximated as aussian, The 
‘ b even for relativel 


approximation can be uite accurate, y small values of N, in the 
central region of the gaussian curve near the mean. However, the approximation 


- 

can be very inaccurate in the lail regions away from the mean, even for large 
Jalues of N (Davenport, 1970; Melsa and Sage, 1973). Of course, the approx- 
imation is made more accurate by increasing N. 


*Equal Distributions 
If all of the statistically independent random variables being summed are contin- 
us and have the same distribution function, and therefore the same density, the 


uo 
proof of the central limit theorem is relatively straightforward and is next 


developed. 
Because the sum Yy = X41 + Xyteo+Xy has an infinite variance as 
N--» a9, we shall work with the zero-mean, unit-variance random variable 


N N 4/2 
Wy = (Yn - wlOry = YX = rofl E23, 
iat = 


. ce peg eg (4.7-2) 
instead. Here we define X and a by 
R,=X alli (4.7-3) 
ot,= 0% alli (4.7-4) 


since all the X, have the same distribution. 
The theorem's proof consists of showing that the characteristic function of 
W, is Uhat of a zero-mean, unit-variance gaussian random variable, which is 


Dy, (2) = EXP (—w?/2) (4.7-5) 


+ These are called lattice-type discrete random variables (Papoulis, 1965). 


. When the possible diserele values of cach random 
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i, from Problem 3-28. If this is proved the density of Wy must be gaussian from 
”"(3.3-3) and the fact that Fourier transforms are unique. The characteristic func- 


tion of Wy is 


wo jo ul 
Dy,(@) = E{ee"*} = el exp { y(Xi- x} | 


Noy i#t 


s (fern [ LET ee |" (4.7-6) 
: JNox | 


The last step in (4.7-6) follows from the independence and equal distribution of 


the X,. Next, the exponential in (4.7-6) is expanded in a Taylor polynomial with 
a remainder term Ry/N: 


bfexp [ Gia, (X,- x} 


jo ) : ( Jo ) (x, — 2) Ba} 
= E<l Xe A Sa eet 
{1+ (aee)e« AVS JN ox 2 a 


= | —(w?/2N) + E[RyJ/N — (47-7) 


where E[Ry] approaches zero as N-—» co (Davenport, 1970, p. 442). On substitu- 
tion of (4.7-7) into (4.7-6) and forming the natural logarithm, we have 


In [My,(w)] = N In (1 — (v?/2N) + ELRyI/N} (4.7-8) 
Since 
2 23 
nd-a=-[pseeze| jzj <1 (4.7-9) 


we identify z with (w?/2N) — E[Rw)/N and write (4.7-8) as 


2 2 
In (Oy,(@)] = —(w?/2) + E[Ry] —- : E - Ae) +ree (47-10) 


IN ON 
so 
tonite (®y,(@)}} = In {im oy (o)} = —w?/2 (4.7-11) 
Finally, we have 
iy Dy, (w) = (4.7-12) 
which was to be shown. ’ 


We illustrate the use of the central limit theorem through an example. 


~ 


LEIS 


owe ERNIE ET, RANIAIM VARKIAULES, ANID RANDOM SIGINAL PRINCIPLES 


ee 


Example 4.7-1 Consider the sum of just two independent uniformly distrib- 
uted random variables X, and X, having the same density 


Sls) =~ Lula) — ule ~ a] 


where a > 0 is a constant. The means and variances of X, and X, are X¥ = 


a/2 and o% = a?/12, respectively. The density of the sum W=X, +X, is . 


available from Example 4.6-1 (with b = a): 


Sw) = Z tri (2) 
a a 


where the function tri (-) is defined in (E-4). The gaussian approximation to 
W has variance oj = 20% = a?/6 and mean W = 2(a/2) =a: 


e7 (w= 0)3)02/3) 


J n(a?/3) 


Figure 4,7-1 illustrates fiy(w) and its gaussian approximation. Even for the 
case of only two random variables being summed the gaussian upprox- 


Approximation to fiy(w) = 


imation is a fairly good one. For other densities the approximation may be . - 


very poor (see Problem 4-63). 


1.0 
Gaussian approximation 


3 gm two awd 
Se twnaitneta) 


Zhe (w) 


) a daw 


Figure 4.7-1 The triangular density function of Example 4.7-1 and its gaussian approximation. 
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PROBLEMS 


4-1 Two events 4 and B defined on a sample space S are related lo a joint sam- 
ple space through random variables Y and Y and are defined by -l = {Vs x} 
and B= {y, < Y s y.}. Make a sketch of the two sample spaces showing areas 
corresponding to both events and the eventA A B={X sxvyy<¥ s ya}. 

4-2 Work Problem 4-1 for the two events A= {x, <XN sx} and B= 
{vi < Y¥ sya}. a 

4:3 Work Problem 4-1 for the (wo events A = {x, <X¥ <x, or xy < V Sy} 
and B= {y, < Y < yo}. . ne 

4-4 Three events A, B, and C satisfy CC BoA and are defined by A= 
{X<x,, Y sy}, B={X Sx, YS yy}, and C={X sx,, Y sy} for two 
random variables X and Y. 

_(a) Sketch the two sample spaces S and S$, and show the regions correspond- 


ing to the three events. 
(b) What region corresponds to the event An Ba C? 


4-5 A joint sample space for two random variables X¥ and Y has four elements 
(1, 1), (2, 2), (3, 3), and (4, 4). Probabilities of these elements are 0.1, 0.35, 0.05, 
and 0.5 respectively. : 
(a) Determine through logic and sketch the distribution function Fy, (x, )'). 
(b) Find the probability of the event {X < 2.5, Y < 6}. 
(c) Find the probability of the event {X < 3}. 
4-6 Write a mathematical equation for I’y, y(x, y) of Problem 4-5. 
~4-7 The joint distribution function for two random variables X and Y is 


Fy lay y) = ulx)u(y)[b = 7 = eo ee] 


where u(-) is the unit-step function and a > 0. Sketch Fy, y(x, )). 
4-8 By use of the joint distribution function in Problem 4-7, and assuming 
a= 0.5 in each case, find the probabilities: 


(a) P{X <1, Y <2} (b) P{0.5 <X < 1.5} 

(ec) P{-15<X52,1<Y <3}. 
4-9 Find and sketch the marginal distribution functions for the joint distribution 
function of Problem 4-5, 


4-10 Find and sketch the marginal distribution functions for the joint distribu- 
tion function of Problem 4-7, 


4-11 Given the function 
Gy ly ¥) = ulxe(y)LE =e ] 


“Show that this function satisfics the first four properties of (4.2-6) but fails the 
fifth one. The function is therefore not a valid joint probability distribution 
function. 
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4-12 Random variables X and Y are components of a two-dimensional random 


vector and have a joint distribution 

0 x<9 or y <0 

xy Osx<! and O<y<iI 
x Osx<! and l<y 

y l<sx and Osy<il 

1 <x and Isy 


(a) Sketch Fy, lx, y). 
(b) Find and sketch the margina 


4-13 Show that the function 


| distribution functions F(x) and Fy(y). 


0 x<y 
Gy, y(X y= ‘ xzy 


alid joint distribution function. (Hint: Use (4.2-6e).] 


efine random variables by: X = “number of 
number of heads on the second toss" (note 


cannot be av 
4-14 A fair coin is tossed twice. D 
hends on the first toss” and Y =“ 
that X and Y can have only the values 0 or 1). 
(a) Find and sketch the joint density function of X and Y. 
(b) Find and sketch the joint distribution function. 
4-15 A joint probability density function is 


1ab O<x<a and 
fx, rl% Y= i‘ elsewhere 


Find and sketch Fy, r(x, ¥). 
4-16 Ifa < bin Problem 4-15, find: 
(a) P(X + Ys 3a/4} (b) P{Y s 2bX /a}. 


4-17 Find the joint distribution function applicable to Example 4.3-1. 
) applicable to Problem 4-5. Write 


O<y<h 


4-18 Sketch the joint density function Sx. v% ¥ 
an equation for fy, y(% y). 

4-19 Determine the joint density and 
Problem 4-7, 

4-20 Find and sketch the joint density func 
Problem 4-12. 

4-21 (a) Finda censtant b ( 
bert <x <a and 


Sx. y(X% y) = ‘0 


is a valid joint density function. 
(b) Find an expression for tl 


both marginal density functions for 


in terms of a) so that the function 
O<y<a 


elsewhere 


he joint distribution function. 


tion for the distribution function in | : 
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4-22 (a) By use of the joint density function of Problem 4-21, find the marginal 


density functions. 
(b) Whatis P{0.5a< Xs 0.75a} in terms of a and b? 


4-23 Determine a constant b such that cach of the following are valid joint 
densily functions: 


3xy O<x<l d 0 
(a) Sx. y(x, y) = y x an <y< b 
0 elsewhere 
bx(1 — 0<x<05 ‘ 0 
(b) Sx, AX y) = { ( y) x and <y <i 
0 elsewhere 
b(x? + 4y? O<\I. 1 , 
0 Sarees 240" y’) sixl< and OS y<2 
0 elsewhere 
* 4.24 Given the function 
(x? + y/8x xt +y? <b 


Sx. y(X, y) = {° 


elsewhere 


* (a) Find a constant b so that this is a valid joint density function. 
(b) Find P{0.5b<X?+Y?s 0.8b}. (Hint: Use polar coordinates in both 


parts.) 
*4.25 On a firing range the coordinates of bullet strikes relative to the target 
bull's-cye are random variables X and Y having a joint densily given by 


= (x2 + y2)/2e2 


Sx, vl% y= 


2no? 


Here o? is a constant related to the accuracy of manufacturing a gun's barrel. 
What value of o? will allow 80% of all bullets to fall inside a circle of diameter 


6 cm? (Hint: Use polar coordinates.) 
4-26 Given the function 
b(x + y)? —-2<x<2 = and ~3 
Sx. % n={ ‘“ ‘ ase 
0 elsewhere 


(a) Find the constant b such that this is a valid joint density function. 
(b) Determine the marginal density functions frlx) and fry). 


4-27 Find the conditional density functions fy(xl yu) SX | ya) Srly|xs), and 
Sy Xa) for the joint density defined in Example 44-1. 


4-28 Find the conditional density function f,(x | y) applicable tu Example 4.4-2. 
4-29 By using the results of Example 4.4-2, calculate the probability of the event 
(Y $2|X =}. 


t 
t 
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ye 

& 4-30 Random variables X and Y are jointly gaussian and normalized if 

c, 

: | 2_2 Gs 

: Te en | _ xin 2pxy + y 

Rep py al; ee 


(a) Show that the marginal density functions are 


1 
fis) = Fee exp (=8*2) fi) = Tea exw ("72 


(Hint: Complete the square and use the fact that r i 
Peers ha the area under a paussian 

(b) Are X and Y statistically independent? 

‘ 4-31 By use of the joint density of Problem 4-30, show that 

fe ! : 

i KA eee | - $= 

: Jnl — p?) PLT 20 = 6) 

4-32 Given the joint distribution function 

H Fx, (x, y) = u(x)uly)CL — 8 = eo + eet) 

a find: 


(a) The conditional density functions f,( 

x(x] Y = y)and fy(y|X = x). 
ee: (b) Are the random variables X and Y statistically independent? 
ee 4-33 For two independent random variables X and Y show that 


PLY <X}= [- F(x) f(x) dx 
or 
py s x)= |" FAddAW) dy 


4-34 Two random variables X and Y have a joint probability density function 


ob 
Ie 
Se. 
§ 


5 

4 —=x7y O<y< 
a Sx, (X,Y) = 16 nate 
0 elsewhere 
E (a) Find the marginal density functions of X and Y. 
(b) Are X and ¥ statistically independent? 
fe es Show, by use of (4.4-13), that the area under f,(y| x) is unity. 
he ‘4-36 Two random variables & and © have the joint density function 


fal, 0) = SOMO AO = PNY yn 


(a) Find P(O< R$ 1,0<@ <n/2}. 


(b) Find fy(r|© = 2). 
(c) Find f,(r|© < 7) and compare to the result found in part (b), and explain 


the comparison. 
4-37 Random variables X and Y¥ have respective density functions 


Sx) = : Cu(x) — u(x — 4)] 


Sly) = buly)e“"” 

where a > 0 and b > 0, Find and sketch the density function of W =X + YX 
and Y are statistically independent. ° 
4-38 Random variables X and Y have respective densily functions 

Sy(x) = O.1d(x — 1) + 0.28(x — 2) + 0.45(x — 3) + 0.30(x - 4) 

Sy) = O.4d(y — 5) ++ 0.586) — 6) + 0.159 — 7) 
Find and sketch the density function of W = NX + YitX and ¥ are independent. 
4-39 Find and sketch the density function of W =X + Y, where the random 
variable X is that of Problem 4-37 with a= 5 and Y is that of Problem 4-38, 
Assume X and Y are independent. 
4-40 Find the density function of W = X + Y, where the random variable X is 
that of Problem 4-38 and Y is that of Problem 4-37. Assume X and Y are inde- 
pendent. Sketch the density function for b = 1 and b = 4. 
#441 Three statistically independent random variables X,, X_, and X, all have 
(he same density function : 


i=1,2,3 


1 
xf a a Cu(x)) - u(x, ~~ d)] 


Find and sketch the density function of Y = X,+X,+X;ifa>od is constant. 


ADDITIONAL PROBLEMS 


4-42 Ina gambling game two fair dice are tossed and the sum-of the numbers 
that show up determines who wins among two players. Random variables VY and 
Y represent the winnings of the first and second numbered players, respectively. 
The first wins $3 if the sum is 4, 5, or 6, and loses $2 if the sum is 11 or 12; he 
neither wins nor loses for all other sums. The second player wins $2 for a sum of 
8 or more, loses $3 for a sum of 5 or less, and neither wins nor loses for other 
sums. 

(a) Draw sample spaces $ and S, and show how elements of § map to ele- 
ments of S,. 

(b) Find the probabilities of all joint outcomes possible in Sy. 


ee 


boii fee 


wand 


aad 
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4-43 Discrete random variables X and Y have a joint distribution function 
Fy lx, ) = O.1Outx + 4)u(y — 1) + O.15u(x + 3)u(y + 5) 
+ 0.17u(x + Luly — 3) + 0.05(x)uly — 1) 
+ O.1Bu(x — 2)u(yr 2) + 0.23u(x — Ajuly — 4) 
+ O.f2u(x — 4)u(y + 3) 
Find: (a) the marginal distributions Fy(x) and Fy(y) and sketch the two functions, 
(b) X and ¥, and (c) the probability P(-1 < X $4, -3<Y <3}. 
4-44 Random variables X and Y have the joint distribution 


(cece 
see 


4 ere — eo) O<x<4 


Fy s(x, y) = 40 x<Oory<0O 


1+ . eo 3? 
Find: (a) The marginal distribution functions of X and Y, and (b) the probability 
P{3<Xs5,1< Y <2}. 

4-45 Find the joint distribution function of the random variables having the joint 
density of Problem 4-48. 

4-46 Find a value of the constant b so that the function 


Sx, (% 9) = bxy? exp (— 2xy)u(x — Duly — 1) 


fer 4<xandany y29 


is a valid joint probability density. 
4-47 The locations of hits of darts thrown at a round dartboard of radius r are 
determined by a vector random variable with components X and Y. The joint 
density of XY and Y is uniform, that is, 

fn? xP? Hy? < r? 

favo = 0 elsewhere 

Find the densities of X and Y. 
4-48 Two random variables X and Y have a joint density 


Serbs ¥) = Ulu) ~ ule — 4yJuty)y? exp [-(e + Dy] 


Find the marginal densities and distributions of X and Y. 
4-49 Find the marginal densities of X and Y using the joint density 


fxr y) = Qu(x)u(y) exp [-(» + 2)| 


4-50 Random variables X and Y have the joint density of Problem 4-49. Find 
the probability that the values of ¥ are not greater than twice the values of X for 


x3. 
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4-51 Find the conditional densities Slx| Y = y) and Sr\X =x) applicable to 
the joint density of Problem 4-47, 
4-52 For the joint density of Problem 4-48 determine the conditional densities 
Sxl ¥ = y)and fy |X = x). 
4-53 The time it takes a person to drive to work is a random variable Y. Because 
of traffic driving time depends on the (random) time of departure, denoted X, 
which occurs in an interval of duration To that begins at 7:30 A.M. each day. 
There is a minimum driving time T, required, regardless of the time of departure. 
The joint density of X and Y is known to be 
Seve y= ey — TP uly - T,)Lu(x) — u(x — To)} exp [-(y — THX + \)) 
where 
c= (1+ Te)?/2E(1 + To)? — 1) 
(a) Find the average driving time that results when it is given that departure 
occurs at 7:30 A.M. Evaluate your result for Ty = 1h. 
(b) Repeat part (a) given that departure time is at 7:30 A.M. plus Tp. 
(c) What is the average time of departure if Ty = 1 h? (Hint: Note that point 
conditioning applies.) 
* 4-54 Start with the expressions 
PLY syoB 


F(y|B) = P{Y < y|B} = P(B) 


dF (y|B 
fyi 8) = SOI 


which are analogous to (4.4-1) and (4.4-2), and derive Fy(y|x, < X S$ x,) and 
Sy xa < X S %) which are analogous to (4.4-15) and (4.4-16). 
*4.§5 Extend the procedures of the text that lead to (4.4-16) to show that the joint 
distribution and density of random variables X and Y, conditional on the event 
B= {y,< Y S yo}, are 


0 YSMe 
Fy yx, y) — Fy le ye 
x, v4 y) x, r(%, Y.) ye SVs 


Fry) — Fre) 
Fy y(x, Ys) — Fx y(X Ya) 


ma Y<y 


Fy(y) — Fy(ya) 


Fy rs Ye < Y<y)= 


and 
0 ysy, and = y>Ye 
Sx. 0% VIN < YS ys) = Sx (ey) 


pee ole ES Fi < < 
Fimo) oF 
4-56 Determine if random variables X and Y of Problem 4-53 are statistically 


independent. 
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4-57 Determine if X and Y of Problem 4-49 are statistically independent. 
4-58 The joint density of four random variables X,,i= 1, 2,3, and 4, is 


4 
Sexa X3, xa%p X20 X35 X4) = I] exp (~2]x,]) 
fw] 


& a 
ea Find densities (a) Seis, x3(%1 Xa, x3 | x4) (b) Sxy.xi(X%1, X2 1x3, Xa), and 
" (c) Sy (1 |¥2, X35 X4) 


the probability density of W. Compare the result with (4.6-5). Is the density still a 
convolution of the densities of X and Y? Discuss. 


4-60 Statistically independent random variables X and Y have respective 
ao densities 
. | Sal) = Culx + 12) — ux = 12) = fx/t217/12 
Lely) = (1/4)u(y) exp (— y/A) 
Find the probabilities of the events: 


(a) {¥ $8 —(2]X|/3)}, and (6) {Y <8 +2] X[/3)}. 
Compare the two results. 


4-61 Statistically independent random variables X and Y have respective 


: densities 

Sale) = Su(x) exp (—5x) 

: Srly) = 2u(y) exp (~2y) 

x 

t Find the density of the sum W = X + Y, 

i *4-62 N statistically independent random variables X;,i=1, 2,..., N, all have 
al the same density 

V 

t Sx{X)) = au(x,) exp (—ax,) 

i where a > 0 is a constant. Find an expression for the density of the sum W = 
t X, + Xp 4-55 + Xy forany N. 

: *4-63 Find the exact probability density for the sum of two statistically indepen- 
i 


dent random variables each having the density 


oo 


Jalx) = 3[u(x + a) = u(x ~ a))x?/203 


where a >0 is a constant, Plot the density along with the gaussian approx- 


imation (to the density of the sum) that has variance 202 and mean 2X. Is the 
approximation a good one? ; 


*4-64 Work Problem 4-63 except assume 
Sx(x) = (1/2) cos (x) rect (x/1). 
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4-59 If the ditlerence W = X — Y 1s tormed instead of the sum in (4.6-1), develop ° 


CHAPTER 


FIVE 


OPERATIONS ON 
MULTIPLE RANDOM VARIABLES 


5.0 INTRODUCTION 


After establishing some of the basic theory of several random variables in the pre- 
vious chapter, it is appropriate to now extend the operations described in Chap- 
ter 3 to include multiple random variables. This chapter is dedicated to these ex- 
tensions. Mainly, the concept of expectation is enlarged to include two or more 
random variables. Other operations involving moments, characteristic functions, 
and transformations are all special applications of expectation. 


5.1 EXPECTED VALUE OF A 
FUNCTION OF RANDOM VARIABLES 


When more than a single random variable is involved, expectation must be taken 
with respect to all the variables involved. For example, if g(X, Y) is some func- 
tion of two random variables X and Y the expected value of g(-,) is given by 


j= Fla, YN) = i ‘3 ale, Me, ross 9) dx dy (5.1-1) 


“Oo yO 


This expression is the two-variable extension of (3.1-6). 
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For N random variables X;, X2, .+-1 Xn and some function of these vari- 
ables, denoted g(X,,.... Xn) the expected valuc of the function becomes 


§ = Elg(X 4, +++ Xa] 


: J ee ee 


oo om 


‘Thus, expectation in general involves an N-fold integration when N random vari- 


ables are involved. 
We illustrate the application of (5.1-2) with an example that will develop an 


important point. 


AT 
Example 5.t-1 We shall find the mean (expected) value of a sum of N 
weighted random variables, If we let 


N 
g(X py v0) Xn) Gir 
#1 


where the “weights” are the constants a, the mean value of the weighted 
sum becomes 


N 
Ela Xs wees Xp) = { Lax,| 


N wo w 
= 5 | of aXe Sess coca Xn(%ty oy Xn) dx, ++* dxy 


from (5.1-2). After using (4.3-8), the terms in the sum all reduce to the form 
| ay 2X) Sx (Xi) aX = Ea, XJ = 4 ECX 1] 
so 
N N 
A yaix,| ae » % ELX] 
t=] pe jet 


which says that the mean value of a weighted sum of random variables equals 
the weighted sum of mean values. 


The above extensions (5.1-1) and (5.1-2) of expectation do not invalidate any 
of our single random variable results. For example, let 


(Xp ees Xn) = X1) (5.1-3) 


= SP ace wie 
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and substitute into (5.1-2). After integrating with respect to all random variables 
except X ,, (5.1-2) becomes 


g = Elg(X1)) = { 


oO 


GX 1)Sx,(%1) 4X3 (5.1-4) 


which is the same as previously given in (3.1-6) for one random variable. Some 
reflection on the reader's part will verify that (5.1-4) also validates such earlier 


* (opics as moments, central moments, characicristic function, etc, for a single 


random variable. 


Joint Moments About the Origin 


Onc important application of (5.1-1) is in defining joint moments about the origin. 
They are denoted by m,, and are defined by 


My, = ELX"Y*) = {" i x"y"fg, ree y) dx dy (5.1-5) 


for the case of two random variables X and Y. Clearly myo = ELX"] are the 
moments m, of X, while mo, = E[Y*] are the moments of Y. The sum n +k is 
called the order of the moments. Thus 1192, M20, and m,, are all second-order 
moments of X and Y. The first-order moments mo, = ELY] = and m= 
E[X] = & are the expected values of Y and X, respectively, and are the coordi- 


nates of the “center of gravity ” of the function fx, r(x, y). 
The second-order moment m,, = E[X Y]} is called the correlation of X and 


Y. It is so important to later work (hat we give it the symbol Ryy. Hence, 


Ryy = My, = ELXY] -| |  xyfy, x, y) dx dy (5.1-6) 


If correlation can be written in the form 
Ryy = ELXJECLY) (5.1-7) 


then X and Y are said to be uncorrelated. Statistical independence of X and Y is 
sufficient to guarantee they are uncorrelated, as is readily proven by (5.1-6) using 
(4.5-4). The converse of this statement, that is, that X and Y are independent if X 
and Y are uncorrelated, is not necessarily true in general. 
If 
Ryy =0 (5.1-8) 


for two random variables X and Y, they are called orthogonal. 
A simple example is next developed that illustrates the important new topic 


of correlation. 


{ Uncorrelated gaussian random variables are, however, known to also be independent (see 
Section 5.3). 
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Example 5.1-2 Let X be a random variabje that has a mean value X = 
E[X] = 3 and variance o2 = 2. From (3.2-6) we easily determine the second 
moment of X about the origin: E[X?] = my) = 02 + X? = 11, 

Next, let another random variable Y be defined by 


Y = ~6X +22 


The mean value of Y is P= ELY] = E[—6X +22] = —6X + 22 =4. The 
correlation of X and Y is found from (5.1-6) 


Rxy = m,, = E[XY] = EL—6X? + 22X] = —6E[X?] + 22% 
= —6(i1) + 22(3) = 


Since Ryy =0, X and Y are orthogonal from (5.1-8). On the other hand, 
Ryy # E[XJELY] = 12, so X and Y are not uncorrelated [see (5.1-7)]. 

We note that two random variables can be orthogonal even though cor- 
related when one, Y, is related to the other, X, by the linear function 
Y = aX + b. It can be shown that X and Y are always correlated if |a| # 0, 
regardless of the value of b (see Problem 5-9). They are uncorrelated if a = 0, 
but this is not a case of much practical interest. Orthogonality can likewise 
be shown to occur when a and b are related by b = —aE[X?]/E[X] when- 
ever E[X] #0, If E[X] = 0, X and Y cannot be orthogonal for any value of 

aexcepta =0,a noninteresting problem. The reader may wish to verily these 
statements as an exercise. 


For N random variables X,, X2,...,Xy, the (ny +g +-++ + ny)-order 
moments are defined by 


Mayng ony = ELXUXD 0 XH] 


={ vf XT XN ey xX tr eer Sy) AX, hs dxy  (5.1-9) 


-0 -@ 


where n,,n,,..., My are all integers = 0, 1,2,.... 


Joint Central Moments 


Another important application of (5.1-{) is in defining joint central moments. For 
{wo random variables X and Y, these moments, denoted by 4,,, are given by 


Bax = EL(X — XY — 1 
= { i. (x — X)'y- Y)\fy. y(x, y) dx dy (5.1-10) 


—“@2 
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The second-order central moments 
Myo = E[(X — XP] = 0} (5.1-11) 
Ho2 = EL(Y — YP] = of (5.1-12) 


are just the variances of X and Y. 
The second-order joint moment j,, is very important. It is called the covari- 
ance of X and Y and is given the symbol Cy,. Hence 


Cry = fy = EX - XY - y) 
= fe { ” = Ry - Y)Sx, lx, y) dx dy (51-13) 


By direct expansion of the product (x — X\y — ¥), this integral reduces to the 
form 


Cyy = Ryy — RP = Ryy — ELXJELY] ¥ (51-14) 


when (5.1-6) is used. If X and Y are cither independent or uncorrelated, then 
(5.1-7) applies and (5.1-14) shows their covariance is zero: 


Cyy =0 X and Y independent or uncorrelated (5.1-15) 
If X and Y are orthogonal random variables, then 
Cyy = ~FELXJELY] X and Y orthogonal (5.1-16) 


from use of (5.1-8) with (5.1-14). GEA Cyy = 0 if cither Y or Y also has zero 
mean value, 
The normalized second-order moment 


P= il Sao Hor = Cxy/ox oy (5.t-1 74) 
given by 
pa Za) athe a (5.1+17b) 
Ox dy 


is known as the correlation coefficient of X and Y. It can be shown (sce Problem 
$-10) that 


“f<epst..~ (5.1-18) 


For N random variables X,, Xz, ..., Xy the (my + ny + e+ + ny)-order joint 


** central moment is defined by 


Hayny oo aAN = EU(X, ve Xi(X, 7 x)" gee (Xn ore X y)""7} 


w th wo ~ 
elec 


(xy — XN) vy cc talX ae cree Mw) UN, oo dy (5.1-19) 


An example is next developed that involves the use of covariances. 
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Example 5.1-3 Again Ict X be a weighted sum of N random variables X;; 
that is, Ict 
N 
X= yo X, 
dwt 
where the «, are real weighting constants. The variance of X will be found. 
From Example 5.1-1, 


N N 
E(X] = Ya A(X) = Ya, X, = X 
tm] t= 
so we have 


N 
Xx-X= ¥ a(X, — X) 
t= 


and 


N N 
oh = BX = Ry} = A] Lats AVY ay x| 
ial =f 
N N N N 
De ay ay EEX) — XX, - X)\J= p> pe Cxix, 
det jal - ” 
Thus, the vatiance of a weighted sum of N random variables X, (weights at) 
equals the weighted sum of all their covariances Cx,x, (weights «,a,). For the 
special case of uncorrelated random variables, where 


0 ifxj 
ill te i=j 


is truc, we gel 


the variance of a weighted sum of uncorrelated random variables 


In words: 
of the random variables 


(weights )) equals the weighted sum of the variances 


(weights a7). 


. 


*5,.2 JOINT CHARACTERISTIC FUNCTIONS 


The joint characteristic function of two random variables X and Y is defined by 


Oy, AO, @) = Eten tier") (5.2-1) 


where w, and cw, are real numbers. An equivalent form is 


@y, (4, @2) = | | Sxy¥% ylelars tor dx dy (5.2-2) 


OD, ~-@ 
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This expression is recognized as the two-dimensional Fouricr transform (with 
signs of w, and w, reversed) of the joint density junction. From the inverse 
Fourier transform we also have 


t «o «© 
Sx. Y) = Gn { { Dy (Wy, Wa)e M41" dew, dw, (5.2-3) 


=o ~c2o 


By setting either w, = 0 or w, = 0 in (5.2-2), the characteristic functions of X 
or Y are obtained. They are called marginal characteristic functions: 


(1) = By, y(w1, 0) (5.2-4) 

©,(w2) = @,, (0, 3) (5.2-5) 

Joint moments m,, can be found from the joint characteristic function as 
follows: 

aby: y(@1) wn) 


a ty 
dw} 004 


Mn = (J ** (5.2-6) 
wy 20,0220 


This expression is the two-dimensional extension of (3.3-4). 


Example 5.2-1 Two random variables X and Y have the joint characteristic 
function 


Dy, y(W4, Wz) = EXP (—2w? — 8«}) 
We show that X and Y are both zero-mean random variables and that they 


are uncorrelated. 
The means derive from (5.2-6): 


RX = E[X] = myo = aj Moy On) 


6a, 01 70,0280 
= —j(—40,) exp (—204 - 809] =0 
' 0120.02 20 
? = ELY] = mo, = —j(— 163) exp (~2u} ~ 809] =0 
Pres wo, =0,02"0 


Also from (5.2-6): 


Ryy = ELXYJ = my, =(-/)? [exp (—2«} — so9)| 


dw, Ow, Lained 


=0 


1 20,0220 


= —(—4w,)(— 16w.) exp (—2w} — 8w}) 


Since means are zcro, Cyy = Rxy from (5.1-14). Therefore, Cyy = 0 and X 
and Y are uncorrelated. 


y 
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The joint characteristic function for N random variables X,, X3, ..., Ny is 


defined by 
Daa, saa XolWty vvey Wy) = ElelaiXt toe + Jone (5.2-7) 
Joint moments are obtained from 
R . 
Myrna ny = (=H) Faust oe Ls (5.2-8) 
where 2 
Ren tn tert ny (5.2-9) 


5.3 JOINTLY GAUSSIAN RANDOM VARIABLES 


density function of two gaussian 
(b) tandom variables. 


, (a) 
ae Gaussian random variables are very important because they show up in nearly 
ee every area of science and engineering. In this section, the case of two gaussian 
ie random variables is first examined. The more advanced case of N random vari- 
he ables is then introduced. 
| Be Two Random Variables 
t oe om 
4 q ‘ Figure 5.3-1 Sketch of the joint 
e 


Two random variables X and Y are said to_be jointly gaussian if their joint 
density function ts of the form 


The locus of constant values of fy (x, y) will be an ellipset as shown in Figure 
5.3-1b. This is equivalent to saying that the line of intersection formed by slicing 
the function fy y(x, y) with a plane parallel to the xy plane is an ellipse. 

Observe that if p = 0, corresponding to uncorrelated X and Y, (5.3-1) can be 


(5.3-1) 


oy ox Oy oF 


[s= Ky? 2plx — RXy — De i 


which is sometimes called the bivariate gaussian density. Here 


z ELX] (53-2) written as 
a . x,y) = Sel Sly) (5.3-8) 
H Y = E[Y] (5.3-3) Sx % Y) HK) | 
E ok = E[(X — X)?] (5.3-4) where f,(x) and f,(y) are the marginal density functions of X and Y given by 
of = EL(Y — 7] (5.3-5) fies |- (x= 4)" os ] (5.3-9) 
p= BUX — RLY — Py/oyay (5.3-6) J2n0% oa 1g 
Figure 5.3-1a illustrates the appearance of the joint gaussian density function nfl Pare aes y- ry (5.3-10) 
(5.3-1). Its maximum is located at the point (X, ¥). The maximum valu is Sly) = angi Pp 202 
obtained from 


Sa. Y) SSy AX, P) = 


tT When ay = ay and p = 0 the cllipse degenerates into a circle; when p = +1 or —1 the ellipses 
degenerate into axes rotated by ungles 1/4 and —n/4 respectively that pass Uhrough the point (2, ). 


aha 


x! Org ogy. 
Bear WES 


(5.3-7) 
2noxoy/1 — p? 
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Now the form of (5.3-8) is sufficient to guarantee that X and Y are statistically 
independent. Therefore we conclude that any two uncorrelated gaussian random 
variables are also statistically independent. It results (hat a coordinate rotation 
(lincar transformation of X and Y) through an angle 


(5.3-11) 


is sufficient to convert correlated random variables X and Y, having variances o} 
and o2, respectively, correlation coefficient p, and the joint density of (5.3-1), into 
two statistically independent gaussian random variables.t 

By direct application of (4.4-12) and (4.4-13), the conditional density func- 
tions fy(x| Y = y) and fy(y| X = x) can be found from the above expressions (sce 
Problem 5-29). 


nnn 
Example 53-1 We show by example that (5.3-11) applies to arbitrary as well 
as gaussian random variables. Consider random variables Y, and Y, related 
to arbitrary random variables X and Y by the coordinate rotation 


Y, = X cas (0) -+ ¥ sin (0) 
Y, = —X sin (0) + ¥ cos (0) 
If. and ¥ are the means of X and Y, respectively, the means of ¥; and Y, 


are clearly ¥, = ¥ cos (0)+ Ysin (0) and ¥, = —X sin (0) + Ycos (0), 
respectively. The covariance of Y; and Y, is 
Cris = EU, - nes a ¥,)) 
= EL{(X — X) cos (li) + (¥ — ¥) sin (0)} 
»{—(X — X) sin (0) + (¥ - Y) cos (()}J 
= (a? — 03) sin (0) cos (f) + Cxy{cos? (0) — sin? (0)] 
= (a3 ~ 02)(4) sin (20) + Cxy cos (20) 


Here Cyy = EL(X — XXY — Y)) = poxoy- If we require Y, and Y, to be 
uncorrelated, we must have Cy,y, = 0. By equating the above equation to 
zero we obtain (5.3-11). Thus, (5.3-!1) applies to arbilrary as well as gaussian 
random variables. 

a ee 


t Wozencraft and Jacobs (1965), p. 153. 


OPERATIONS ON MULTIPLE RANDOM VARIABLES 127 


*N Random Variables 


N random variables X,, X32. ---) Xw are called jointly gaussian if their joint 


B «. density function can be written ast 


Neal {- be = RNG k= ai} 


Sie, coon XnX 9 v0 Xn) = (n)"? 
(5.3-12) 
where we define matrices 
x, -X, : 
[x- R=] x, (5.3-13) 
Xy—- Xp 
and 
Ci, Ci, Cin 
(Cyl = ge a na (8.14) 


Cur Cnr *** Cww 


We usc the notation [-]' for the matrix transpose, [:]7! for the matrix inverse, 
and |[-]] for the determinant. Elements of [Cy], called the covariance matrix of 
the N random variables, are given by 


a3 i=j 
C,, = E[L(X,— 8X, —- X) =| a wes (5.3-15) 
iJ { L J Dd Cyyx, i #j 
The density (5.3-12) is often called the N-variate gaussian density function. 
For the special case where N = 2, the covariance matrix becomes 
a pox Ox 
C 1-[ ie is 5.3-16 
PFx,Fx, ox, 
so 
1 1/03, —lox,ox 
[Cx]7! = | : ee (5.3-17) 
WG —p)L-ploxex, Veoh, 
HICxl7 "| = 1/0%,0%,(1 — 0”) (5.3-18) 


On substitution of (5.3-17) and (£.3-18) into (5.3-12), and letting X, = X and 
X, = Y, it is easy to verify that the bivariate density of (5.3-1) results. 


¢ We denote a matrix symbolically by use of heavy brackets [-}. 
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_ *Some Properties of Gaussian Random Variables 


“We State without proof some of the properties exhibited by N jointly gaussian 
“random variables X,,..., Xy- 


1, Gaussian random variables are completely defined through only their first- 
and second-order moments; that is, by their means, variances, and covari- 
ances. This fact is readily apparent since only these quantities are needed to 
completely determine (5,3-12). 


2. If the random variables are uncorrel ey_are also statistically indepen- 


dent, This property was given earlier for two variables. 


3. Random variables produced by a linear transformation of X,, wie Xy Will also 


be gaussian, as proven in Section 5.5. 
4. Any k-dimensional (k-variate) marginal density function obtained from the N- 


dimensional density function (5.3-12) by integrating out N — k random vari- 
ables will be gaussian. If the variables are ordered so that Xj, ..., X, occur in 
the marginal density and X,4,,..., Xy are integrated out, then the covariance 
matrix of X,..., X, is equal to the leading k x k submatrix of the covariance 
. matrix of X,,..., Xy (Wilks, 1962, p. 168). 
5. The conditional densit Xp eves Kal Xnts = Xntyy cer Xy = Xy) is 
gaussian (Papoulis, 1965, p. 257), This holds for any k < 


*5.4 TRANSFORMATIONS OF 
MULTIPLE RANDOM VARIABLES 


The function g in either (5.1-1) or (5.1-2) can be considered a transformation 
involving more than one random variable. By defining a new variable Y = 
WX, Xz, ..., Xy), we see that (5,1-2) is the expected value of Y. In calculating 
expected values it was not necessary to determine the density function of the new 
random variable Y. It may be, however, that the density function of Y is required 
in some practical problems, and its determination is briefly considered in this 
section. : 


In fact, one may be more gencrally interested in finding the joint density 


function for a set of new random variables 
A TT 


YX Xai ae Kel Pe Qin (5.4-1) 


defined by functional transformations Tj. Now all the possible cases described in 


Chapter 3 for one random variable carry over to the N-dimensional problem, 
That is, the X, can be continuous, discrete, or mixed, while the functions T, can 
be linear, nonlinear, continuous, segmented, etc. Because so many cases are 
possible, many of them being beyond our scope, we shall discuss only one repre- 


sentative probiem, 


We shall assume that the new random variables Y,, given by (5.4-1), are pro- 


duced by single-valued continuous functions T having continuous partial deriv- 
‘ 


eer avers 


OPERATIONS ON MULTIPLE KANIIUM VARINA 6s 


t is further assumed thal a set of inverse continuous functions 
das single-valued contin- 


atives everywhere, 


T; exists such that the old variables may be expresse 


nous functions of the new variables: 
hous uncuons OO 


: 2 
X= TM Yar Yn) el (5:42) 


These assumptions mean that_a point in the joint sample space of the X; maps 

i es Yj. 

into only one point in the space of the new variabl 
et KR, be a closed region of points in the space o the X, and Ry be the cor 

responding region of mapped points in the space of the Y), then the probability 

that a point falls in Ry will equal the probability that its mapped point falls in 

Ry. These probabilitics, in terms of joint densities, are given by 


ie | fort ee | Xy) dx ae dX y 


Rx 
_ [-- [ fom yi» wey Yn) dy; a dyy (5.4-3) 


Ry 


This equation may be solved for Fev ow ra vey Yy) by treating it as simply a 

iple integral involving a change of variables, ae 
see aes on the ett side of (5.4-3) we change the variables xj to new fees 
ables y, by means of the variable changes (5.4-2). The integrand is change by 
direct functional substitution. The limits change from the region Ry to the on 
Ry. Finally, the differential hypervolume dx,s dxy will change to - se ue 
[J] dy, «++ dyy (Spiegel, 1963, p. 182), where |J| is the magnitude of (he jaco- 


biant J of the transformations. The jacobian is the determinant of a matrix of 
derivatives defined by 


(5.4-4) 


Thus, the left side of (5.4-3) becomes 


[-- | Fan xylXar even Sy) Ny dN 


Rx 


~ je [Foon tbe = Ti ety THM dy yy (5-4-5) 


Ry 


‘ + After the German mathematician Karl Gustav Jakob Jacobi (1804-1851). 
ae ae é 
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Since this result must cqual the right side of (5.4-3), we conclude that 
Lecce Year eves Yn) = Sey tn = Ty ee Xn = Tr')lJ1 (54-6) 


When N = 1, (5.4-6) reduces to (3.4-9) previously derived for a single random 


variable. 
The solution (5.4-6) for the joint density of the new variables Y, is illustrated 


here with an example. 


ae nun ono nr 


Example 5.4-1 Let the transformations be linear and given by 
Y¥, = T,(X4, X2) = aX, + bX, 
Y, = T,X, X,)=cX, + dX, 


where a, 6, ¢, and d are real constants. The inverse functions are casy to 
obtain by solving these two equations for the two variables X, and X,: 


X,=T7'(%, Ys) = d¥%, — bY ad — be) 
X,=T7'(% Ys) = (—c¥, + a¥,)ad — he) 
where we shall assume (ad — bc) # 0. From (5.4-4): 


df(ad — be) ~—bf(ad — bc) 
—cef(ad — he) af(ad — be) 


1 


— 


dm * (ad — be) 


Finally, from (5.4-6), 


a 
; ui ad—be' ad—be 
fire Ia) = Tad bey 


nner a 


*5,5 LINEAR TRANSFORMATION OF 
GAUSSIAN RANDOM VARIABLES 


Equation (5.4-6) can be readily applied to the problem of linearly transforming a 
set of gaussian random variables Xi 


(5.3-12) applies. The new variables Y,, Yy..0+5 


nv are 
Vos ayyNy bay Nab anNXn 


Yy = 0X + 0g) X_ +0 + aw Xn (5.5-1) 


Yy = Oy X1 + dy, Xq “htt hyn Xn 


Xy, ..6) Xy for which the joint density of 


os 
Bs 5 
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where the coefficients a,,i andj = 1, 2,..., N, are real numbers. Now if we define 
the following matrices: 


Q;, 442 ain 
ital oe (5.5-2) 
he es ia 
Y, y, Xx, x, 
\Yl=]: [¥] = IX}=]: iXp=] : 
Yy Yy Xn Ry 
(5.5-3) 
then it is clear from (5.5-1) that ' 
1Y) = (TH) 1Y — MP=ITHX — XI (5.5-4) 
[XP=ITIMYE = (xX — RS ITV WY - Yj (5.5-5) 
so long as [T] is nonsingular. Thus, 
X= TP (My ees Ye) = aly, talV¥, +000 + al" Yy (5.5-6) 
or = a = q') (5.5-7) 
X,-X,=a'(¥,- P)te tal%(Yy — Py) (5,5-8) 


from (5.5-5). Here a’! represents the ijth element of [T]7'. 

The density function of the new variables Y;,..., Yvy is found by solving the 
right side of (5.4-6) in two steps. The first step is to determine |J|. By using 
(5.5-7) with (5.4-4) we find that J equals the determinant of the matrix [TI7!. 
Hence,t i 


"4 


Jl = |ITI'Y = 5.5-9 
The second step in solving (5.4-6) procecds by using (5.5-8) to obtain 
N N : 
Cryx, = EU(X1 — RMX) — R= Lak ¥ aEl(N — LY — Y.) 
kul m=} 
N N 
= Day aCny, (5.510) 


ae mal 


Since Cy,x, is the jth element in the covariance matrix [Cy] of (5.3-12) and Cry, 


t We represent the magnitude of the determinant of a matrix by CDi. 
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is the knith element in the covariance matrix of the new variables ¥,, which we 
denote [Cy], (5.5-10) can be written in the form 


Cb = (TI 'ICyITY)! (S.5-11) 
Here [T]' represents the transpose of [7]. The inverse of (5.5-11) is 
Cy) = ETHIC TY (5.5-12) 
which has a determinant 
WCxd'f = Uy PTI P (5.5-13)° 


On substitution of (5.5-13) and (5.5-12) into (5.3-12): 
Frcs sna XX SP ics) Xw = Ty!) 


— UTH [Cy en {- x ~ XMITHC HTH — XI 


Oar ; } (5.5-14) 


Finally, (5.5-14) and (5.5-9) are substituted into (5.4-6), and (5.5-4) is used to 
obtain 


. Cnt ~ PICT - 
hie eugae Ze exp {- W= Theol 'y= FI 


Gay ; (5.5-15) 


This result shows that the new random variables Y,, Y),..., Yy are jointly gauss- 
Jan because (5,5-15) is of the required form, 
In summary, (5.5-15) shows that a linear transformation of gaussian random 


variables produces gaussian random variables, The new variables have mean, 
pc A A Rak SC aL 

values 

CL 


N 
Y= YayX, (S.5-16) 
k=l 


from (5.5-!) and covariances given by the elements of the covariance matrix 


[Cy] = [THCyHT]! (5.5-17) 


as found from (5.5-11). 


Sarena 
Example 5.5-1 Two gaussian random variables X, and X, have zero means 
and variances of, = 4 and a2, = 9, Their covariance Cy,x, equals 3. If X, 
and X, are linearly transformed to new variables Y, and Y, according to 

Y, = Xx, = 2X; 

%=3N,4+4X, 


we use the above results to find the means, variances, and covariance of Y 
and Y. 
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Ilere 


ip os 4 3 
{T] = E a and |CyJ = 309 


Since X, and X, are zero-mean and gaussian, Y, and Y, will also be zero- 
mean and gaussian, thus YP) = Qand Y, = 0. From (5.5-17): 


“) -24 3 1 ‘l=[ 28 ee 
Ie =ITHcaTY =| 5 a 9}L-2 4) [-66 252 


Thus, of, = 28, 07, = 252, and Cy,y, = — 66. 


*5.6 COMPLEX RANDOM VARIABLES 


A complex random variable Z can be defined in terms of real random variables X 
and Y by 
: Z=X+jY (5.6-1} 
where j = ./—1, In considering expected values involving Z, the joint density of 


X and Y must be used. For instance, if y(-) is some function (real or complex) of 
Z, the expected value of g(Z) is obtained from 


wo wo 
E[y(Z)} = | { lz) fx, Oy) dx dy (5.6-2) 
~% J-@ 
Various important quantities such as the mean value and variance arc 
obtained through application of (5.6-2). The mean value of Z is 
Z = E[Z] = E[X] + jELY] =X +jY (5.6-3) 
The variance o2 of Z is defined as the mean value of the function g(Z) = 
|Z — E[Z]|’; that is, 
of = EC |Z ~ E[Z] |") (5.6-4) 
Equation (5.6-2) can be extended to include functions of two random 
variables 
Zm = Xin + 5% (5.6-5) 
and 
Z,= NX, 45%, (5.6-0) 
n £m, if expectation is taken with respect to four random variables Kms Yous Nn 
Y, through their joint density function fy, y,, x. r(%mo me nv Ja) HE this density 
satisfies 


Sen Yu Xas y(%m > Yrs Xas Vn) = fx, YnlXm : Vm Sx,, ¥,(%n ’ Yn) (5.6-7) 


wan peed 
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then Z,, and Z, are called statistically independent. The extension to N random 
variables is straightforward, 
The correlation and covariance of Z,, and Z, are defined by 


Rigas = ELZ Za) nem (5.6-8) 
and 
Crate = UZ an Te EUZ AZ, a HUZANI nAm (5.6-9) 


respectively, where the superscripted asterisk* represents the complex conjugate. 
If the covariance is 0, Z,, and Z, are said to be uncorrelated random variables. By 
sctting (5.6-9) to 0, we find that 

Rzz, = ELZSJE(Z0] mezN (5.6-10) 
for uncorrelated random variables. Statistical independence is sufficient to guar- 


antec that Z,, and Z, are uncorrelated. 
Finally, we note that two complex random variables are called orthogonal if 


their correlation, given by (5.6-8), equals 0. 


PROBLEMS 


6-1 Random variables X and Y have the joint density 
| 
_ 0<x <6 and 

Se. 9) = {24 


it) elsewhere 


O<y<4 


What is the expected value of the function g(X, Y) =(X Y)?? 


5-2. Extend Problem 5-! by finding the expected value of g(X 1, X2, X,,X4) = 
NUNGNVANT, where my, Mas My, and ng are integers 20 and 


| 
TF O<x, <aand0 <x, <band0 <x3 <¢ 


abed 
Sar tacds XX te X29 X30 Xa) = and OQ <x, <d 
0 " elsewhere 
5-3 The density function of two random variables X and Y is 
" Fras 9) = rdodu(y) 1604” 


Find the mean value of the function 


! aed 
5 0<Xs35 and 0<Ys5 


X = | ! 
ax. ¥) ~-! 5 <X and/or 3 <Y 


0) all other X and Y 


Co EE SESE BREA EEE GES SERB T ML 
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5-4 For the random variables in Problem 5-3, find the mean value of the 
function 


ah Vee wee 
5-5 Three statistically independent random variables X,, X,,and X, have mean 
values %, = 3, &, =6, and X, = —2. Find the mean values of the following 
functions: 
(a) g§Xy, X_. Xs) =X, + 3K, + 4Xy 
(b) (Xp. Xa, X3) = XiXaX3 
(c) y(X4, X_. X3) = —2XX_— 3X Xa + 4X,X; 
(d) G(X, X2,X3) =X, t+ Xi t+%Xs 
5-6 Find the mean value of the function 
g(X, Y) =X? + Y? 


where X and Y are random variables defined by the density function 
= (x2 + y2)/202 


Sx. r(x, y) = 2 


2na 


with a? a constant. 
5-7 Two statistically independent random variables X and Y have mean values 
R = E[X] =2 and P = E[Y] = 4. They have second moments X? = E[X?] =8 
and Y? = E[Y?] = 25. Find: 

(a) the mean value (b) the second moment and 

(c) the variance of the random variable W = 3X — Y. 


8 Two random variables X and Y have means X = 1 and Y = 2, variances 
ot = 4 and o} = I, and a correlation coefficient pyy = 0.4. New random variables 


WV and V are defined by 
V=—X+2Y W=X+3Y 


Find: 
(a) the means (b) the variances (c) the correlation and 
(d) the correlation coefficient pyw of V and W. 


5-9 Two random variables X and Y are related by the expression 
Y=aX +b 


where a and b are any real numbers. 
(a) Show that their correlation coefficient is 


m I ifa> 0 for any b 
Oe Teg ifa <0 for any b 


(b) Show that their covariance is 
Cry = aay 


wiheie gi is the variance of X. 


‘ 
~ 


i Te, ea a 
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r ; : 
5-10 Show that the correlation coefficient satisfies the expression 


H 
Ip} =u < 
V Ho2 #20 


5-11 Find all the second-order moments and 
function given in Problem 5-3, 


! 


5-12 Random variables X and Y have the joint density function 


2 
firs = fOF? (40 -l<x<1 and 
0 elsewhere 


-3<y<3 


(a) Find all the second-order moments of X and Y, 
(b) What are the variances of X and Y? 
(c) What is the correlation coefficient? 


5-13 Find all th ird- ; 
Problem 5-12, ¢ third-order moments by using (5.1-5) for X and Y defined in 


5-14 For discrete random variables X and Y, show that: 
(a) Joint moments are 


N M 
Ma = py >» P(x,, y)xt yt 


(6) Joint central moments are 
N M 
Hat = Pp» Py Plxi, yx, — XY", — FP) 


where P(x;, y)) = P{X =x,, Y= X has N i 
navi wey, 1 yy; as N possible values x,, and Y has M 


5-15 For two random variables X and Y: 
Sx, (% y) = 0.155(x + 1)d(y) + 0.15(x)d(y) + 0.16(x)5(y ~ 2) + 0.46(x — 1)d(y + 2) 
+ 0.26(x — 1I)5(y ~ 1) + 0.058(x — 1)d(y — 3) 


Find: (a) th i : : 
of X and Me i correlation, (b) the covariance, and (c) the correlation coefficient 


(d) Are X and Y either uncorrelated or orthogonal? 
5-16 Discrete random variables X and Y have the joint density 
fix, r(x, ¥) = 0.48(x + a)5(y — 2) + 0.35(x — a)d(y — 2) 
+ O1d(x — a)5(y - a) + 0.25(x — 1)d(y — 1) 


Determine the value of a, if any, that minimizes the correlation between X and Y 
and find the minimum correlation. Are X and Y orthogonal? 


central moments for the density 
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5-17 For two disercte random variables X and Y: 
Se. lx, y) = 0.30(x — a) 5(y — @) + 0.500 + a)S(y — 4) + 0.28(x + 2)d(y + 2) 


Determine the value of a, if any, that minimizes the covariance of X and Y. Find 
the minimum covariance. Are X and Y uncorrelated? 


§-18 The density function 


x , 
. <x <2 and VO<yp<3} 
Se. v ¥) = 9 . 
0 elsewhere ‘ 


applies to two random variables X and Y. 
(a) Show, by use of (5.1-6) and (5.1-7), that X¥ and Y are uncorrelated. 
(b) Show that X and Y are also statistically independent. 


5-19 Two random variables Y and Y have the density function 


ae = (x + 05y) O<x<2 ~~ and O<y<3 
xX, Y) = 


0 elsewhere 


(a) Find all the first- and second-order moments. 
(b) Find the covariance. 
(c). Are X and Y uncorrelated? 


§-20 Define random variables V and W by 
V=X+ayY 
W=X-aY 


where a is a real number and XY and Y are random variables, Determine a in 
(erms of moments of X and Y such that ¥ and W are orthogonal. 

*5.21 If X and Y in Problems 5-20 are gaussian, show that FY and V are sta- 
listically independent if a? = a3/o}, where a2 and of are the variances of X and 
Y, respectively, 

§-22 Three uncorrelated random variables X,, X;, and X3 have means XY, = 1, 

X¥,= —-3, and ¥,= 1.5 and second moments E[X?] = 2.5, ELX3] = 11, and 
ELN2] = 3.5. Let Y =X, — 2X, + 3X, be a new random variable and find: 

(a) the mean value, (b) the variance of Y, 


§-23 Given WW = (aX +3Y)? where X and Y are zero-mean random variables 
with variances 02 = 4 and oj = 16. Their correlation coefficient is p = —0.5. 

(a) Find a value for the parameter a that minimizes the mean value of W. 

(b) Find the minimum mean value. 


*5.24 Find the joint characteristic function for X and Y defined in Problem 5-3, 
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*5.25 Show that the joint characteristic function of N independent random vari- 


ables X,, having characteristic functions Dy (ww) is 


N 
Dyson xO eer On) = [] ®x(o) 
t=i 


*5.26 For N random variables, show that 
Lg agltDre cee Mn SP yey yg es O) = I 


*6.27 For two z¢ro-ican gaussian random variables X and Y, show that their 
joint characteristic function is 


Wy, y(coy, a) = EXP {— "lox wy + 2pay ay o,@, + of wi} 


*6.28 Zero-mean gaussian random variables X and Y have variances o} = 3 and 
ai = 4, cespectively, and a correlation coefficient p = — A. 
(a) Write an expression for the joint density function. 
(b) Show that a rotation of coordinates through the angle given by (5.3-11) 
will produce new statistically independent random variables. : 
46.29 Vind the conditional density functions SAXLY = y) and HOTX = x) appli- 
cable to two gaussian random variables X and Y defined by (5.3-1) and show 
that (hey are also gaussian. 


*8.30 Zcro-mean gaussian random variables X;, X2, and X; having a covariance 
matrix 


4 2.05 1.05 
[Cy] =] 208 4 2.05 
105 205 4. 


are transformed to new variables 
Y, = 5X, +2X,-%3 
Y, = —X,+3X,+X3 
¥, = 2X, --N24+2%5 


(a) Find the covariance matrix of Y,, Y,,and ¥3. 
(b) Write an expression for the joint density function of Y,, Y,,and Y;. 


*5.31 A complex random variable Z is defined by 
Z = cos (X) +j sin (Y) 


where Y and Y are independent real random variables uniformly distributed 
from —m lo R,. 

(a) Find the mean valuc of Z. 

(hb) Find the variance of Z. 


<t 
19 
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ADDITIONAL PROBLEMS 


5.32 Two random variables have a uniform density on a circular region de- 
fined by 
nr? xt ty? <r? 
Sr. 10 Y= {t elsewhere 


Find the mean value of the function g(X, Y) =X? + y?, 
£5.33 Define the conditional expected valuc of a function g(X, Y) of random vari- 
ables X and Y as 


E[g(X, Y)| B] = { { Ox, WSx, 1% y| B) dx dy 

(a) If event B is defined as B = {ve < Y S yp}, where y, < yy are constants, 
evaluate E{g(X, Y)| B). (Hint: Use results of Problem 4-55.) 

(b) If B is defined by B = {Y = y} what does the conditional expected value 
of part (a) become? : 
5.34 For random variables X and Y having Rel, P=2, 0} =6, 07 = 9, and 
pm — 74, find (a) the covariance of X and Y, (b) the correlation of X and Y, 
and (c) the moments m9 and mo. : 
5-35 Ra',, X?=4,, P=2, Y? ='%,, and Cyy = —1/2,/3 for random vari- 
ables X and Y. 

(a) Find 0%, 07, Rxy, and p. 

(b) What is the mean value of the random variable W =(X + 3Y)? + 
2X + 37 
5-36 Let X and Y be_statistically independent random variables with X = 71, 
Xi=4, Pel, and Y?=5.:For a random variable W= X —2Y +1 find 
(a) Ryys(b) Rew s(c) Ryws and (d) Cyy.(e) Are X and Y uncorrelated? 


5-37 Statistically independent random variables X and Y have moments mio = 


2, gq = 14, mo2 = 12, and mm, = — 6, Find the moment jiz2. 
5-38 A joint density is given as 
x(y + 1.5) O<x<1 and O<y<lI 
Sx 1 ¥) ‘0 elsewhere 


Find all the joint moments m,,, and k=0,1,.... 
5.39 Find all the joint central moments H,,, 7 and k = 0, 1,..., for the density of 
Problem 5-38. 
*5.40 Find the joint characteristic function for random variables X and Y de- 
fined by 
Sy. (X,Y) = (1/2n) rect (x/n) rect ((x + y)/m] cos (x + y) 


Use the result to find the marginal characteristic functions of X and Y. 
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*5-41 Random variables X, and X, have the joint characteristic function 
Py, x,(y, @2) = [(1 — j2w XI ~ j2w2)}-"? 


where N > 0 is an integer. 
(a) Find the correlation and moments mgq and tg, . 
(b) Determine the means of X, and X,. 
(c) What is the correlation coefficient? 


*5-42 The joint probability density of two discrete random variables X and Y 
consists of impulses located at all lattice points (mb, nd), where m = 0, I, ..., M 
and n= 1, 2,..., N with b > 0 and d > 0 being constants. All possible points are 
equally probable. Determine the joint characteristic function. 


*5-43 Let X,,k = 1, 2,..., K, be statistically independent Poisson random vari- 
ables, each with its own variance b, (Problem 3-16). Show that the sum X = 


Xp + Xy tere + X xq is wv Poisson random variable. (Hint: Use results of Prob- 
lems 5-25 and 3-53.) 


5-44 Assume oy = oy =a in (5,3-1) and show that the locus of the maximum of 
the joint density is a line passing through the point (X, 7) with slope 1/4 (or 
—7/4) when p = I (or ~1). 

5-45 Two gaussian random variables X and Y have variances 02 = 9 and a= 
4, respectively, and correlation coefficient p, It is known that a coordinate rota- 


tion by an angle —7/8 results in new random variables Y, and Y; that are uncor- 
related. What is p? 


*5-46 Let X and Y be jointly gaussian random variables where o? =o} and 
p= —\. Find a (ransformation matrix such that new random variables Y, and Y, 
are statistically independent, 


*$-47 Random variables X and Y having the joint density 
Sx, rl y) = (A)u(x — 2)uly — 1)xy? exp (4 = 2xy) 
undergo a transformation 
1 H 
IT] = ; - a 


to generate new random variables Y, and Y,. 
(a) Find the joint density of Y, and Y,. 


(6) Show what points in the y,y2 plane correspond to # nonzero value of the 


new density. 


*5-48 Equation (5.4-5) can sometimes be used to find the density of a single func- 
tion of several random variables if auxiliary random variables are used. Apply the 
idea to finding the density function of Z = aX Y, where a is a constant and X and 
Y are random variables, by defining the auxiliary variable W = X. 


*5-49 Apply the method of Problem 5-48 to finding the density function of 
Z = bY/X, with ba constant, when using the auxiliary variable W = X. 


7 PVIM PIT Se ttt es cas 


: . - are define e mean and 
+ #5.) Two gaussian random variables X, and 4 are defined by the mean ¢ 


‘BE. covariance matrices ' 
Moe 3 ans 
x -| | ICxl = : 


-7J/5 4 


i are : i ansformation 
Two new random variables Y, and Y) are formed using the tran i 


im: 
: Pe Find matrices («) [¥] and (b) {Cyl. (c) 
i and Y3. 


Also find the correlation coefficient of Y, 


n variables Z, and Z, have zero means. The correlation of 


d Z, is 4, while the correlation of the imaginary parts is 6. 
: art of Z, are statistically independent as 
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RANDOM PROCESSES 


6.0 INTRODUCTION 


In the real world of engineering and science, it is necessary that we be able to ” 


deal with time waveforms. Indeed, we frequently encounter random time wave- 
forms in practical systems. More often than not, a desired signal in some system 
is random. For example, the bit stream in a binary communication system is a 
random message because each bit in the stream occurs randomly. On the other 
hand, a desired signal is often accompanied by an undesired random waveform, 
noise. The noise interferes with the message and ultimately limits the performance 
of the system. Thus, any hope we have of determining the performance of systems 
with random waveforms hinges on our ability to describe and deal with such 
waveforms. In this chapter we introduce concepts that allow the description of 
random waveforms in a probabilistic sense. 


6.1 THE RANDOM PROCESS CONCEPT 


The concept of a random process is based on enlarging the random variable 
concept to include time. Since a random variable X is, by its definition, a func- 
lion of the possible outcomes s of an experiment, it now becomes a function of 
both s and time. In other words, we assign, according to some rule, a time 


function 
x(t, 5) (6.1-1) 


to every outcome s. The family of all such functions, denoted X(t, 5), is called a 
random process. As with random variables where x was denoted as a specific 
value of the random variable X, we shall often use the convenient short-form 


142 
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notation x(t) to represent a specific waveform:of a random process denoted by 
X(t). 

Clearly, a random process X(t, s) represents a family or ensemble of time 
functions when t and s are variables. Figure 6.1-1 illustrates a few members of an 


© ensemble. Each member time function is called a sample function, ensemble 


member, or sometimes a realization of the process. Thus, a random process also 
represents a single time function when ¢ is a variable and s is fixed at a specific 
value (outcome). 
A random process also represents a random variable when ¢ is fixed and s is 
a variable. For example, the random variable X(t), 8) = X(t,) is obtained from 
the process when time is “frozen” at the valuc t;. We often use the notation X, 
to denote the random variable associated with the process X(/) at time {,. X , cor- 
. responds to a vertical “slice” through the ensemble at time f,, as illustrated in 
Figure 6.1-1, The statistical properties of X, = X(t,) describe the statistical 
properties of the random process at time ¢,. The expected value of X, is called 
the ensemble average as well as the expected or mean valuc of the random proccss 
(at time ¢,). Since t; may have various valucs, the mean value of a process may 
not be constant; in general, it may be a function of time. We casily visualize any 


Figure 61-1 A continuous 
random process. (Reproduced 
from Peebles (1976) with per- 
mission of publishers Addison- 
e Wesley, Advanced Book Pro- 
Py gram.) 
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ae of random variables X, derived from a random process X(#) al times ¢,, 
f=1,2,...: 


X,= X(t), 5) = X(t) (6.1-2) 
F . random process can also represent a mere number when ¢ and s are both 
ixed, 
Classification of Processes 


It is convenient to classify random processes according to the characteristics of ¢ 
and the random variable X = X(t) at time ¢, We shall consider only four cases 
based on rand X having values in the ranges -00o <t < o and -w <x< oo.t 


+ Other cuses cun be defined based on a definition of random Processes on 4 finite time interval 


(see for example: Rosenblatt (1974), p. 91; Prabhu (1965), p. 1; Miller (1974), p. 31; Parzen (1962), °° 4 


p. 7; Dubes (1968), p. 320; Ross (1972), p. 56). Other recent text : | 
(1985) ahd Ora ues Davloroe tigeey exis on Badly nroressrs are Helstrom 


Figure 6.1-2 A discrete random process formed by heavily limiting the waveforms of Figure 6.1-1. 


(Reproduced fr Pevhles vi erinissi y i 
ae from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book 
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If X¥ is continuous and ¢ can have any of a continuum of values, then V() is 
called a continuous random process. Figure 6.1-1 is an illustration of this class of 
process, Thermal noise generated by any realizabie network is a practical 
example of a waveform that is modeled as a sample function of 4 continuous 
random process. In this example, the network is the outcome in the underlying 
random experiment of selecting a network, (The presumption is that many net- 
works are available from which to choose; this may not be the case in the real 
world, but it should not prevent us from imagining a production line producing 
any number of similar networks.) Each network establishes a sample function, 
and all sample functions form the process.t . 

A second class of random process, called a discrete random process, corre- 
sponds to the random variable X having only discrete values while ¢ is contin- 
uous. Figure 6.1-2 illustrates such a process derived by heavily limiting the 
sample functions shown in Figure 6.1-1, The sample functions have only two dis- 


t Note that finding the mean value of the process at any dime fis equivalent to finding the average 
vollage that would be produced by all the various networks at time t 


Xana) 


Figure 6.1-3 A cominuous rine 
dom sequence formed by sam- 
pling the waveforms of Figure 
6.11, [Reproduced fram Peebles 


. (1976), with permission — of 


e publishers: Addison-Wesley, Ad- 
%* vanced Book Program.) 
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Figure 61-4 A discrete random sequence formed by sampling the waveforms of Figure 6.-2. 
(Adapted from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book Program.) 


crete values: the positive level is generated whenever a sample function in Figure 
6.1-1 is positive and the negative level occurs for other times. 

A random process for which X is continuous but time has only discrete 
values is called a continuous random sequence (Thomas, 1969, p. 80). Such a 
sequence can be formed by periodically sampling the ensemble members of 
Figure 6.1-1. The result is illustrated in Figure 6.1-3. 

A fourth class of random process, called a discrete random sequence, corre- 
sponds to both time and the random variable being discrete. Figure 6.1-4 illus- 
trates a disercte random sequence developed by sampling the sample functions of 
Figure 6.1-2. 

In this text we are concerned almost entirely with discrete and continuous 
random processes. 
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Deterministic and Nondeterministic Processes 


In addition to the classes described above, a random process can be described by 
the form of its sample functions. If future values of any sample function cannot be 
predicted exactly from observed past values, the process is called nondeterministic. 
‘The process of Figure 6.1-1 is one example, 3 

A process is called deterministic if future values of any sample function can be 
predicted from past values. An example is the random process defined by 


X(t) = A cos (wot + ©) (6.1-3) 


Here A, ©, or wo (or all) may be random variables. Any one sample function cor- 
responds to (6.1-3) with particular values of these random variables. Therefore, 
knowledge of the sample function prior to any time instant automatically allows 
prediction of the sample function's future valucs because its form is known. 


6.2 STATIONARITY AND INDEPENDENCE 


As previously stated, a random process becomes a random variable when time is 
fixed at some particular value. The random variable will possess statistical 
properties, such as a mean value, moments, variance, etc., that are related to its 
density function. If two random variables are obtained from the process for two 
lime instants, they will have statistical properties (means, variances, joint 
moments, etc.) related to their joint density function. More generally, N random 
variables will possess statistical propertics related to their N-dimensional joint 
density function. 

Broadly speaking, a random process is said to be stationary if all its sta- 
tistical properties do not change with time. Other processes are called nonsta- 
tionary, These statements are not intended as definitions of stationarity but are 
meant to convey only a general meaning. More concrete definitions follow. 
Indecd, there are several “levels” of stationarity, all of which depend on the 
density functions of.the random variables of the process. 


Distribution and Density Functions 


To define stationarity, we must first define distribution and density functions as 
they apply to a random process X(t). For a particular time f,, the distribu- 
tion function associated with the random variable X, = X(t,) will be denoted 
Fy(x1; ¢,). It is defined ast 


Fy(xyi ty) = P{X(4)) S xy} (6.2-1) 


t F(x,5 t,) is known as the first-order distribution function of the process X(t). 
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i i . To prove (6.2-9), we find mean values of the random variables ¥, = V(t.) and 
tion function of one random variable. Only the notation has been altered to xX, : X(tz). For X,: 


. for any real number *,. This is the same definition used all along for the distribu- 
e reflect the fact that it is Possibly now a function of time choice t,. 


a For two random variables X¥, = X(t;) and X, =X i 

as ° . ‘6 = = t 1, th - igi, = = 4 . oy sd 

a distribution function is the two-dimensional extension of ean Seas ee BLO ty - fale aes ee 
Pryxys X25 ty, ta) = P{X(t,) < x4, X(t) < x,} (6.2-2) For X;: 


In a simi nat 
na similar manner, for N random variables X,= X(t), i= 1,2,..., N, the Neh X Syler3 fa) dxit (6.2-11) 


order joint distribution function is 


E[X,] = E[X(t,)] = { 


Now by letting t, =, + A in (6.2-11), substituting (6.2-8), and using (6.2-10), 


Pret ay rey SW 5 Cy very by) = PLX(G) S Xt ves X(ty) S xy} (6.2-3) we get 


Joint density functions of interest are found from i ivati 
the above three relationships:t Pees derivatives:of E(X(t, + O)) = ELX(t,)) (6.2-12) 
e Slvii th) = dF x,; 1 dx, (6.2-4) which must be a constant because ¢, and A are arbitrary. 
e SX, X25 yy ty) = 8 Fylx,, x20, dx F) 
i Srtx Nuit N pepe eee. (6275) Second-Order and Wide-Sense Stationarity 
B. MA yy ery Xyy ty veey ty) =O PX yy oes XN 5 Cay wees ty MX, +++ Oxy) (6.2-6) be ; | 
‘ P< A process is called stationary to order two if its second-order density function 


satisfies 


Cm 


mds 
a 


Statistical Independence 


X25 lay bg) =Se(X1, X25 by + A, ty + A) (6.2-13) 

Two process sige tepteee 4 ; . Li%y X25 bas be xX 

poi x rte xt Rca ar artaaindl : We as wana for all t,,¢,, and A. Afler some thought, the reader will conclude that (6.2-13) is a 
F Broup Y(ti), Y(ta), sy Yin) function of time differences 1, —1t, and not absolute time (let arbitrary A = —1,), 


for any choice of times t,, t te 
an 1 Fay coos Ens yy Ob, wey by. Independenc i 
the joint density be factorable by groups: eve ‘ ee 


Fa.v%r, reer Kye Vis seer Yas ty eery In» ty tee, to 


=X, 0, Xn bay eens wh weer Vag Cy voay a) (6.2-7) 


First-Order Stationary Processes 


A random process is called stationary to order one if its first-order density func- 
{ion does not change with a shift in time origin. In other words 


: S013) = Sen; t, + d) (6.2-8) 


must be true for any f, and any real nun i i . 
: . ‘ nber A if X(t) is lo be a first-order sta- 
tionary process, () rst-order sta 


Consequences of (6.2-8) are that Jx(X15)) is i 
: i i (15 ndependent i 
Process mean value E[X(t)] is a constant: =o ‘ ey ee as 


E[X(t)] = X = constant (6.2-9) 


t Analogous to distribution functions 


. these are culled irst-, second-, und Nth-order density Litt Sod 
: ive} 
fi . O u J fi 


A second-order stationary process is also first-order stationary because the 
second-order density function determines the lower, first-order, density. 

Now the correlation E[X ,X,] = E[X(t,)X(t,)) of a random process will, in 
general, be a function of t, and ¢,. Let us denote this function by Ryy(ty, f2) and 
call it the autocorrelation function of the random process X(t): 

Ryxllyy ty) = ELX (0) X (4). (6,2-14) 
A consequence of (6.2-13), however, is that the autocorrelation function of a 
second-order stationary process is a function only of time differences and not 
absolute time; that is, if 


cy ea (6.2-15) 


then (6.2-14) becomes 
Ryxlty, ty +t) = ELX (ty) X(t, + 2)) = Ryx(t) (6.2-16) 


Proof of (6.2-16) uses (6.2-13); it is left as a reader exercise (sec Problem 6-6). 
Many practical problems require that we deal with the autocorrelation 
function and mean value of a random process. Problem solutions are greatly 


t Note that the variable x, of integration has been replaced by the alternative variable x, for con- 
venience. 
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simplified if these quantities are not dependent on absolute time. Of course, 
second-order stationarity is sufficient to guarantee these characteristics. How- 
ever, it is often more restrictive than necessary, and a more relaxed form of sta- 
tionarity is desirable. The most useful form is the wide-sense stationary process, 
defined as that for which two conditions are true: 


E[X(t)] = X = constant 
ELX(NX(t + 1)} = Ryx(t) 


(6.2-17a) 
(6.2-17b) 


A process stationary to order 2 is clearly wide-sense stationary, However, the 
converse is not necessarily (ric, 


TT 


Example 6.2-1 We show that the random process 
X(t) = A cos (Wot + O) 


is wide-sense stationary if it is assumed that A and ap arc constants and © is 
a uniformly distributed random variable on the interval (0, 2m). The mean 
value is : 
2 


ELX(t)] = { A cos (Wot + 0) = dO = 0 


0 
The autocorrelation function, from (6.2-14) with t, =f and t, =¢+1, 
becomes 


Ryy(t, (+ t) = ELA cos (wot + O)A cos (WWgt + Wot + ©) 
A 
= Efcos (ug t) -b COS (2g C++ Wg t + 20)) 


At At ' ’ 
oe COS (M9 T) = Ff cos (2g t + Mot + 20)] 
The second term casily evaluates to 0. Thus, the autocorrelation function 
depends only on t and the mean value is a constant, so X(t) is wide-sense 
stationary. 
ee ee a 
When we are concerned with two random processes X(t) and Y(0), we say 
they are jointly wide-sense stationary if each satisfies (6.2-17) and their cross- 
correlation function, defined in general by 


Ryyltyy to) = ELX(t1) ¥(t2)] (6.2-18) 
is a function only of time difference t = f, — ¢; and not absolute time; that is, if 
Ryyplt, £4 t) = ELX(N Y(t + 9] = Rey(t) (6.2-19) 
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" N-Order and Strict-Sense Stationarity 


By extending the above reasoning to N random variables X, = X(t), i= 1,2,..., 
N, we say a random process is stationary to order N if its Nth-order density func- 
tion is invariant to a time origin shift; that is, if 


SX iy ces Xu Cpe nes Ow) HSC, ees ln Gp tA, ey ty +A) (6.2-20) 


for al! t,,..., fy and A. Stationarity of order N implics stationarity to all orders 
k<N. A process stationary to all orders N =1, 2, ..., is called strict-sense 
Stationary, 


Time Averages and Ergodicity 


The time average of a quantity is defined as 


j Tr 
AC-}= lim == | [J dt (6.2-21) 

: T-20 2T -T 
Here A is used to denote time average in a manner analogous to E for the sta- 
tistical average. Time average is taken over all time because, as applicd to 
random processes, sample functions of processes are presumed to exist for all 

lime. : : 

Specific averages of interest are the mean value x = A[x(t)] of a sample func- 
tion (a lower case letter is used to imply a sample function), and the time autocor- 
relation function, denoted R,,(t) = A[x(t)x(t + t)]. These functions are defined by 


é al, thay 
x = A[x(t)] = ae oT [ ) dt (6.2-22) 
RyAt) = ALX(QX(l -F t)] 
eae ae a 
= id oT [ mt +1) dt (6.2-23) 


lor any one sample function of the tprocess X(1), these last (wo integrals 
simply produce two numbers (for a fixed value of t). However, when all sample 
functions are considered, we see that x and ‘R,,(t) are actually random variables. 
By taking the expected value on both sides of (6.2-22) and (6.2-23), and assuming 
the expectation can be brought inside the integrals, we obtaint 


E[X] = X (6.2-24) 
E[R,.(t)] = Ryx(t) (6.2-25) 


Now suppose by some theorem the random variables X and R,,(t) could be 
made to have zero variances; that is, X and R,,(t) actually become constants. 


¢ We assume also that X(1) is a stationary process so that the mean and the autocorrelation func- 
tion are not time-dependent. 
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Then we could write 
x= (6.2-26) 
Ryx(t) = Ryy(t) (6.2-27) 


In other words, the time averages % and Ny.(t) equal the statistical averages ¥ 
and Ry,(t) respectively. The ergodic theorem allows the validity of (6.2-26) nnd 
(6.2-27). Stated in loose terms, it more generally allows all time averages to equal 
the corresponding statistical averages. Processes that satisfy the ergodic theorem 
are called ergodic processes, ; 
Ergodicity is u very restrictive form of stationarity and it may be difficult to 
prove that it constitutes a reasonable assumption in any physical situation. 
Nevertheless, we shall often assume a process is ergodic to'simplify problems. In 
the real world, we are usually forced to work with only one, sample function of a 
process and therefore must, like it or not, derive mean value, correlation func- 
tions, etc. from the time waveform. By assuming ergodicity, we may infer the 
similar statistical characteristics of the process. The reader may feel that our 
theory is on shaky ground based on these comments. However, it must be 
remembered that all our theory only serves to model real-world conditions, 
Therefore, what difference do our assumptions really make provided the assumed 
model does truly reflect real conditions? 
Two random processes are called jointly ergodic if they are individually 
ergodic and also have a time cross-correlation function that equals the statistical 
cross-correlation function:t 


R(t) = lim { " aly + 0) de = Ryv(t) (6.2-28) 
T70 2T -T ; 


6.3 CORRELATION FUNCTIONS 


The autocorrelation and cross-correlation functions were introduced in the pre- 
vious section. These functions are examined further in this section, along with 
their properties. In addition, other correlation-type functions are introduced that 
are important to the study of random processes, 


Autocorrelation Function and Its Properties 


Recall that the autocorrelation function of a random process X(t) is the correla- 
tion E[X ,X,] of two random variables X, = X(t,) and X, = X(t,) defined by 
the process at times ¢, and 1. Mathematically, 


Ryxlly, ta) = ELX(t,)X(t2)] (6.3-1) 


T As in ordinary stationarity, there are various orders of ergodic stationarity. For more detail on 
ergodic processes, the reader is referred to Papoulis (1965), pp. 323-332, 
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For time assignments, f, =f and (, =f, - 1, with oa real number, (62-1) 
assumes the convenient form 
Ryyl, 6-4 1) = ELX(OX( + 0) (6.3-2) 
If X(.) is at least wide-sense stationary, it was noted in Section G2 Meh 
Ryy(t, 6+ t) must be a function only of time difference t=, —¢,. Thus, for 
wide-sense stationary processes 


Rxx(t) = ELX()X(t + 1)] (6.3-3) 

For such processes the autocorrelation function exhibits the following-propertics: 
(1) | Ryx(t)] Ss Ryx(0) (6.3-4) 

(2) Ryx(—t) = Ryx(t) (6.3-5) 

(3) Ryx(0) = ELX7()] (6.3-6) 


The first property shows that Ry,(t) is bounded by its value at the origin, MDH 
the third property states that this bound is equal to the mean-squared value 
called the power in the process. The second property indicates that an autocorre- 
lation function has even symmetry. 

Other properties of stationary processes may also be stated [see Cooper and 
McGillem (1971), p. 113, and Melsa and Sage (1973), pp. 207-208]: 


(4) I ELX(0)] = X # Oand X(t) has no periodic components then 
lim Ryy(t) = X? (6.3-7) 
[thw 


(5) If X(t) has a periodic component, then Ryx(t) will have a periodic com- 
ponent with the same period. 7 (6,3-8) 
(6) If X(t) is ergodic, zero-mean, and has no periodic component, then 


lim Ryx(t) = 0 (6.3-9) 
|] 00 
(7) Ry x(t) cannot have an arbitrary shape. (6.3-10) 


Properties 4 through 6 arc more or less self-explanatory. Property 7 simply says 
that any arbitrary function cannot be an autocorrelation function. This fact will 
be more apparent when the power density spectrum is introduced in Chapter 7, It 
will be shown there that Ry y(t) is related to the power density spectrum through 
the Fourier transform and the form of the spectrum is not arbitrary, 


Example 6.3-1 Given the autocorrelation function for a stationary process is 


(t) = 2 
Rys(t) = 25 + 9 


we shall find the mean value and variance of the process X(t), From property 
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4, the mean value is ELX(t)] = X = /25 = +5. The variance is given by 
(3.2-6), so 


a} = ELX*(0)) — (ELX(9))? 
But ELX2()] = Ryx(C) = 25 + 4 = 29 from property 3, so 


ot =29—25=4 
eS ee te ee 


Cross-Corrclation Function and Its Properties 


The cross-correlation function of two random processes X() and Y(t) was defined 
in (6.2-18). Setting ¢, = ¢and t =f, — f,, we may write (6.2-18) as 


Ryy(t, €+ 0) = ELX(N V(t + 1)) (6.3-11) 
if X(t) and Y(t) are at least jointly wide-sense stationary, Ryy(t, (+ 1) is indepen- 
dent of absolute time and we can write 

Ryy(t) = ELA) Y(t + t)] (6,3-12) 
If 
Ryy(t, f+ t) = 0 (6,3-13) 


then X(t) and Y(t) are called orthogonal processes. If the two processes are sta- 
listicatly independent, the cross-correlation function becomes 


Ryy(t, ¢ + t) = ELX(QELY( + 2) (6.3-14) 

If, in addition to being independent, X(!) and Y(t) are at least wide-sense station- 
ary, (6.3-14) becomes 

Ryy(t) = XY (6.3-15) 


which is a constant. 
We may list some properties of the cross-correlation function applicable to 


processes that are at least wide-sense stationary: 
(1) Ryyl—t) = Rya(t) (6.3-16) 
se (2) [Ravel s /RexORrx(0) (6.317) 
(3) [Rart)l s ALRxx(0) + Rrr(0)] (6.3-18) 


Property | follows from the definition (6.3-12). It describes the symmetry of 
Ryy(t). Property 2 can be proven by expanding the inequality 


EC{ Y(t + 1) + aX(N)}?] 20 (6.3-19) 


where a is a real number (see Problem 6-27). Properties 2 and 3 both constitute 
bounds on the magnitude of Ryy(t). Equation (6.3-17) represents a tighter bound 
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than that of (6.3-18), because the geometric mean of two positive numbers cannot 
excced their arithmetic mean; that is 


JSRxx0)Ryv(0) S ALRxx(0) + Rrv(0)] (6.3-20) 


Example 6.3-2 Let two random processes X(t) and Y(t) be defined by 
X(t) = A cos (Wot) + B sin (wo!) 
Y(t) = B cos (wot) — A sin (wo!) 
where A and B are random variables and wo is a constant. It can be shown 
(Probicm 6-12) that X(t) is wide-sensc stationary if A and B are uncorrelated, 
zero-mean random variables with the same variance (they may have different 
density functions, however). With these same constraints on A and B, Y(t) is 
also wide-sense stationary. We shall now find the cross-correlation function 
Ryy(t, (+ t) and show that X(t) and Y(t) are jointly wide-sense stationary. 
By use of (6.3-11) we have 
Ryylt, +t) = ELX()Y(¢ + 1)J 
= ECAB cos (a9 t) Cos (Wot + Wt) 

+ B* sin (wo) COs (Wo! + Mot) 

— A? cos (wot) sin (wot + wo t) 

— AB sin (wot) sin (Wolf + 9 t)) 

= E[AB] cos (2wot + Wt) 

+ E[B?] sin (wot) cos (wot + Wot) 

— E[A?] cos (wo t) sin (Wot + Wot) 
Since A and B are assumed to be zero-mean, uncorrelated random variables, 
E[AB] = 0. Also, since A and B are assumed to have equal variances, 
E{A?] = E[B?] = o? and we obtain 

: Ryylt, t +t) = —a? sin (wot) 
Thus, X(t) and Y(t) are jointly wide-sense stationary because Ryylt, 6 + 1) 
depends only on t. 
Note from: the above result that cross-correlation functions are not 


necessarily even functions of t with the maximum at t = 0, as is dhe case with 
autocorrelation functions. 


es 
Covariance Functions 


The concept of the covariance of two random variables, as defined by (5.1-13), 
can be extended to random processes. The autocovariance function is defined by 


Cult, (+) = EL{X(0) — ELX(ON}{X(¢ + 2) — ELX(E + I} (63-21) 
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which caa also be put in the form 
Cyx(t, t+ 1) = Ryx(t, 0+) E(X (1) ELX(t + 1)] (6.3-22) 
The cross-covariance function for two Processes X(t) and Y(t) is defined by 
Cyl 6+ 0) = EL{X() — ELX(D)} (YO + 9) ECY(¢+1)}}] — (6.3-23) 
or, alternatively, 
Cyr(t, 0+ 1) = Ryyl(t, t+) - ECX(QJELY (t + 1)] (6.3-24) 
For processes that are at least jointly wide-sense stationary, (6.3-22) and 
(6.3-24) reduce to 
Cyx(t) = Ryx(t) — X? : (6.3-25) 
and : 
Cyy(t) = Ryy(t) — XP ; (6.3-26) 


The variance of a random process is given in general by (6.3-21) with r = 0. 
For a wide-sense stationary process, variance does not depend on time and is 
given by (6,3-25) with r =0: 


o% = EC{X(t) — ELX(t))}?] = Ryy(0) — X? (6.3-27) 
For two random processes, if 
Cyy(t, 6 +1) =0 (6.3-28) 
they are called uncorrelated. From (6.3-24) this means that 
Ryylt, ¢ + 1) = ELX(Q)ELY(t + 0] (6.3-29) 


Since this result is the same as (6.3-14), which applies to independent processes, 
we conclude that independent processes are uncorrelated. The converse case is 
not necessarily (rue, although it is true for jointly gaussian processes, which we 
consider in Section 6.5, ae 


4 


6.4 MEASUREMENT OF CORRELATION FUNCTIONS 


In the real world, we can never measure the true correlation functions of two 
random processes X(t) and Y(t) because we never have all sample functions of the 
ensemble at our disposal. Indeed, we may typically have available for measure- 
ments only a portion of one sample function from each process, Thus, our only 
recourse is to determine time averages based on finite time portions of single 
sample functions, taken large enough to approximate true results for ergodic pro- 
cesses. Because we are able to work only with time functions, we are forced, like 
it or not, to presume that given processes are ergodic. This fact should not prove 
too disconcerting, however, if we remember that assumptions only reflect the 
details of our mathematical model of a real-world situation. Provided that the 
model gives consistent agreement with the real situation, it is of little importance 
whether ergodicity is assumed or not. 
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HT yy 


F fi Rolt, + 27) 
1 


[ Product | 


Figure 6.4-f A time cross-correlation function measurement system, Autocorrelation function mea- 
surement is possible by connecting points A and B and applying either x(/) or y{0). 


Figure 6.4-1 illustrates the block diagram of a possible system for measuring 
the approximate time cross-correlation function of two jointly ergodic ronson 
processes X(t) and Y(t). Sample functions x(t) and y(t) are delayed by amounts 7 
and T — 1, respectively, and the product of the delayed waveforms is formed. 
This product is then integrated to form the output which equals the integral at 
time {, + 27, where ¢, is arbitrary and 2T is the integration period. The inte- 
gralor can be of the integrate-and-dump variety described by Peebles (1976, 
P. 361). . . ons on 
If we assume x(t) and y(t) exist at least during the interval —T <1 and (, is 
an arbitrary time except 0 < 1;, then the output is easily found to be 


1 utt 
Rift, + 2T) = — x(Qy(t + 1) dt (6.4-1) 
2 1 ty -T 
Now if we choose ¢, = Of and assume T is large, then we have 
T 
R,(2T) = 5 x()y(t + t) dt © R(t) = Ryy(t) (6.4-2) 
-T 


Thus, for jointly ergodic processes, the system of Figure 6.4-1 can approximately 
measure their cross-correlation function (t is varied to obtain the complete 


function). 
Clearly, by connecting points A and B and applying cither x(t) or y(t) lo the 
system, we can also measure the autocorrelation functions Ry y(t) and Ryy(t). 


Examiple 6.4-1 We connect points A and B together in Figure 6.4-1 and use 
the system to measure the autocorrelation function of the process X(f) of 
Example 6.2-1, From (6.4-2) 
i 
R,QT) = a | A? cos (wat + 0) cos (Wot + 0 + wot) dt 
-T 


2°T 5 
= cle [cos (Wot) + cos (2wyt + 20 + wo t)] dt 
47 - 
-T 
In writing this result 0 represents a specific value of the random variable 0; 


t Since the processes are assumed jointly ergodic and therefore jointly stationary, the integral 
(6.4-1) will cend to be independent of ¢, if T is large enough. 
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the value that corresponds to the specific ensemble member being used in 
(6.4-2). On straightforward reduction of the above integral we obtain 
R,(2T) = Ryx(t) + 7) 
where 
Ryx(t) = (A?/2) cos (9 1) 
is the true autocorrelation function of X(0, and 
sin (2Wo T) 
2wW_9 T 
is an error term. If we require the error term’s magnitude to be at least 20 
times smaller than the largest value of the true autocorrelation function then 
Je T)| < 0.05Ryx(0) is necessary. Thus, we must have 1/29 T < 0.05 or 
T = 10/wo 
In other words, if T = 10/wo the error in using Figure 6.4-1 to measure the 


autocorrelation function of the process X(t) = A cos (Wot + 9) will be 5% 
or less of the largest value of the truc autocorrelation function. 


«(T) = (A?/2) cos (wot + 20) 


6.5 GAUSSIAN RANDOM PROCESSES 


A number of random processes are important enough to have been given names. 
We shall discuss only the most important of these, the gaussian random process. 

Consider a continuous random process such as illustrated in Figure 6.1-1 
and define N random variables X, = X(t) Xe = X(t Xn = X(ty) corre- 
sponding to N time instamts ty, -+-9 fry sees ty. If, for any N = 1, 2,... and any 
times f,, -.++ fy» these random variables are jointly gaussian, that is, they have a 
joint density as given by (5.3-12), the process is called gaussian. Equation (5.3-12) 
can be written in the form 


oe? {-(1/2)lx = Ryicy be — XV} 
Jan) Cx 


where matrices [x — RX] and [Cyl are defined in (5.3-13) and (5.3-14) and (5.3-15), 
respectively. The mean values X, of X(t,) are 


Xx, = E[Xi) = E[X(t)] (6.5-2) 
The elements of the covariance matrix [Cy] are 
Ca = Crm = E(x. - RMX A X)) 
= EL{X(t) — EEX) X(t) — E(X(t))}) 
= Cyxltir te) (6.5-3) 


(6.5-1) 


Ly(Xpy ere Nn bay cee En 


which is the aulocovariance of X(t) and X(t) from (6.3-21). 
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From (6.5-2) and (6.5-3), when used in (6.5-1), we see that the mean and aulo- 
covariance functions are all that are necded to completely specify a gaussian 
random process. By expanding (6.5-3) to get 


Cxxltis Ge) = Ryxxltis ) — ECX(t)JELX(t)] (6.5-4) 


we sec that an alternative specification using only the mean and autocorrelation 
function Ryx(t), 4) is possible. 

If the gaussian process is not stationary the mean and autocovariance func- 
tions will, in general, depend on absolute time. However, for the important case 
where the process is wide-sense stationary, the mean will be constant, 

R,= ECX(t)] =X — (constant) (6.5-5) 
while the autocovariance and autocorrelation functions will depend only on time 
differences and not absolute time, 

Cyxltis te) = Cxxlte - t) (6.5-6) 
Ryxltis ty) = Raxlte — 4) (6.5-7) 
It follows from the preceding discussions that a wide-sense stationary gauss- 


ian process is also strictly stationary. 
We illustrate some of the above remarks with an example. 


’ 


ies 


Example 6.5-1 A gaussian random process is known to be wide-sense sta- 
tionary with a mean of X = 4and autocorrelation function 


Ryx(t) = 25e7 3! 


We seek to specify the joint density function for three random variables X(¢,), 
i = 1, 2, 3, defined at times t; = fo + (i — 1)/2], with f) a constant, 
Here t, — t; = (k — i)/2,i and k = 1, 2, 3, so 


Rill, — ) = 2507 MP 
and 
Cyxlty — 0) = 25e7 24 N? — 16 


from (6.5-4) through (6.5-7). Elements of the covariance matrix are found 
from (6.5-3). Thus, 
(25 — 16) (25e7>/? — 16) (25e~ *? — 16) 
[Cx] =] (25e7*? — 16) (25 — 16) (25e7>/? — 16) 
(25e~ 9/3 — 16) (25e7 7? — 16) (25 — 16) 


and X, = 4 completely determine (6.5-1) for this case where N = 3. 


weep efee TS 
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Two random processes X(t) and Y(t) are said to be jointly gaussian if the 
random variables X(¢,), ..., X(t), Y(ti), ...» Y(t) defined at times tyy ees, ty for 
X(t) and times 4, ..., thy for Y(0), are jointly gaussian for any N, ¢,,...,tv, M, 0, 
deaailags 


*6.6' COMPLEX RANDOM PROCESSES 


if the complex random variable of Section 5.6 is generalized to include time, the 
result is a complex random process Z(t) given by 


Z(t) = X(t) + JY (0) (6.6-1) 


where X(t) and Y(t) are real processes. Z(t) is called stationary if X(t) and Y(t) are 
jointly stationary, If X(¢) and Y(t) are jointly wide-sense stationary, then Z(1) is 
said to be wide-sense stationary. 

Two complex processes Z,(t) and Zt) are jointly wide-sense stationary if 
each is wide-sense stationary and their cross-correlation function (defined below) 
is a function of time differences only and not absolute time. 


function, and autocovariance function to include complex processes, The mean 
value of Z(t) is 


E(Z(t)) = ELX(t)] + JECY (0) (6.6-2) 

Autocorrelation function is defined by 
Redlt, t + t) = ELZ*()Z(t + 1)] (6.6-3) 
where the asterisk * denotes the complex conjugate. Autocovariance Sunction is 


defined by 
Cralt, ¢ +t) = EL{Z(t) — ELZ())}*{Z(t +t) — ELZ(t + ]}] (66-4) 


If Z(t) is at least wide-sense Stationary, the mean value becomes a constant 


| 2 Z=X+jP (6.6-5) 
Ee. and the correlation functions are independent of absolute time: 

: Raglt, (+1) = Ret) (6.6-6) 
i Cyzlt, t + t) = Cz2(t) (6.6-7) 
For two complex processes Z(t) and Z At), cross-correlation and cross-. 

7 covariance functions are defined by 
| : Razft, 0+ 1) = ELZPQZ {At + 0d] ifj (6.6-8) 
so and 
iE Caralts t+ 1) = ELZ() — ELZ (OY) *(Zfe +) — EZ e+ Mj 
Le (6.6-9) 


KANDOM TOC bsats Lot 


os bfeereakt aig tahe lnts 
respectively. If the two processes are at least jointly wide-sense stationary, 


btain 
: (6.6-10) 


(6.6-11) 


Raz lls 6+ 1) = Ry2{t) i#j 
Cuz Al 0+ t) = Cy z(t) ifj 


Z{t) and Z{0) are said to be uncorrelated processes if Cyy(t, 0+ 1) = 0,14). 
1 4 : — * . 
They are called orthogonal processes if Rzz(t, 0+ 1) = 0,6 #). 


We may extend the operations involving process mean value, autocorrelation 


1 < 
Example 6.6-1 A complex random process V(t) is comprised of a sum of N 


complex signals: 
N 
igs 
vit)= YA, elertl 
ans] 


Here wy/2z is the (constant) frequency of each signal. A, is a randen) aie 
representing the random amplitude of the nth signal. Similily, ; i i 
random variable representing a random phase angle. We assume all ¢ : ee 
ables A, und ©,, for n= 1, 2,..., N, are statistically independent and the ° 
are uniformly distributed on (0, 22). We find the autocorrelation function o 


V(t). 
From (6.6-3): 


Rylt, t+ t) = ELV *(NV(t + OD) 


N N 
- - t + JBm 
‘= ef y A,e Jwot - JO« y A,, ef" Jewot ] 


net mai 


NOON 
= LY eMrelA, Agen” n'] = Ryylt) 
nelmel 
From statistical independence: 
N N 
Ryy(t) = giv » > ECA, An) ELexp {in a ©,))] 
ax=tm=t 
However, 


ELexp {i(®,, — ®,)}] = Ecos (©,, — ©,)} +JELsin (©,, — ©,)] 


_ ie Ie aa [cos (0,, — 0,) +f sin (On — 0,)) dO, dO, 
lo Jo «(20 : 


0 mAn 
mil m=n 
so x a 
Ryy(t) = el" YA 
aat 3 
SS ey ee ae) 
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PROBLEMS 


6-1 A random experiment consis(s of 


RANDOM SIGNAL PRINCIPLES 


selecting a point on some city street that 


has two-way automobile traffic. Define and classify a random process for this 
experiment that is related (o traffic flow. 
6-2 A 10-meter section of a busy downtown sidewalk is actually the platform ofa 


scale that produces a voltage proport 
scale at any time. 


ional to the total weight of people on the 


(a) Sketch a typical sample function for this process. 
(b) What is the underlying random experiment for the proccss? 


(c) Classify the process. 


*@-3 An experiment consists of measuring the weight IV of some person cach 10 
minutes. The person is randomly mate or female (which is not known though) 
with equal probability. A two-level discrete random process X(t) is generated 


where 


X(t) = +10 


The level ~—10 is gencrated in the period following a measurement if the mea- 


sured weight does not exceed Wy (s 
weight exceeds Wo. Let the weight of 
gaussian density 


1 
fyow| male) = anti3 


Similarly, for women 


! 
fGw| female) = Tins 
26.8 


ome constant), Level +10 is generated if 
men in kg be a random variable having the 


exp [-(w- 77,1)? /2(11.3)7] 


exp [—(w - 54,4)?/2(6.8)7) 


(a) Find 1% so that P{W > We | malc} is equal to P(Ws Wo | female}. 


(b) If the levels + 10 are interpret 


ed as “decisions” about whether the weight 


measurcment of a person corresponds to a male or female, give a physical signifi- 


cance to their generation. 


(c) Sketch a possible sample function. 


6-4 The two-level semirandom binary 


X(t) = Aor -A 


where the levels A and —A occur wit 
and n=0, £1, £2... 


process is defined by 
(n-I)T <t< nT 


h equal probability, T is a positive constant, 


(a) Sketch a typical sample function. 


(b) Classify the process. 
(c) Is the process deterministic? 


6-5 Sample functions in a discrete random process are constants; that is 


X() = 


C = constant 
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where C is a discrete random variable having possible values ¢, = 1, ¢2 = 2, and 
¢ = 3 occurring with probabilities 0.6, 0.3, and 0.1 respectively. 

(a) Is X(t) deterministic? 

(b) Find the first-order density function of X(f) al any time f. 
6-6 Utilize (6.2-13) to prove (6.2-16). 
*6-7 A random process X(t) has periodic sample functions as shown in Figure 
P6-7 where B, T, and 4 < T are constants but cis a random variable uniformly 
distributed on the interval (0, 7). 

(a) Find the first-order distribution function of X(t). 

(b) Find the first-order density function. 

(c) Find ECX()]), E[X2(t)], and of. 


0 Ble t ttle e+T t 


Figure P6-7 


*6.8 Work Problem 6-7 for the waveform of Figure P6-8. Assume 2to < T. 


Figure 1°6-8 
*6.9 Work Problem 6-7 for the waveform of Figure P6-9. Assume 419 S T. 
xt : 


tHlalf-cycle 


toby ttle 


Figure P6-9 
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6-10 Given the random process 
X(t) = A sin (wot + ©) 

where A and (ug ure constants and @ is a random variable uniformly distributed 
on the interval (—z, 2). Define a new random process Y(t) = X%(1), 

(a) Find the autocorrelation function of Y(t). 

(b) Find the cross-correlution function of X(t) and Y(¢). 

(c) Are X(t) and Y(t) wide-sense stationary? 

(d) Are X(t) and Y(t) jointly wide-sense stationary? 
6-11 A random process is defined by 


Y(t) = X(t) cos (wot + ©) 


where X(t) is a wide-sense stationary random process that amplitude-modulates a 
carricr of constant angular frequency wo with a random phase © independent of 
X(t) and uniformly distributed on (—z, n). : 

(a) Find ELY(t)}. 

(0) Find the autocorrelation function of Y(t). 

(c) Is Y(t) wide-sense stationary? 


6-12 Given the random process 


X(t) = A cos (wot) + B sin (Wo t) 


where Wo is a constant, and A and B are uncorrelated zero-mean random vuri- 
ables having different density functions but the same variances a7, Show that X(t) 
is wide-sense stationary but not strictly stationary. 
6-13 If X(t) is a stationary random process having a mean value E[X(t)] = 3 and 
autocorrelation function Ryy(t) = 9 + 2¢7!!, find: 

(a) the mean value and 

(b) the variance of the random variable 


: 2 
Y -| X(t) dt 
0 


(Hint: Assume expectation and integration operations are interchangeable.) 
6-14 Define a random process by 


X(t) = A cos (nt) 


where 4 is a gaussian random variable with zero mean and variance 04, 
(a) Find the density functions of X(0) and X(1) 
(b) Is X(0) stationary in any sense? 

6-15 For the random process of Problem 6-4, calculate: 


(a) the mean value ELX()] (b) Ryx(t, = OST, t, = 0.77) (e) Ryxlty = 
0.27, 4, = 127). 


6-16 A random process consists of three sample functions X(t, s,) = 2, X(t, s,) = 
2 cos (t), and X(t, s,) =3 sin (t), each occurring with equal probability. Is the 
process stationary in any sense? 
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6-17 Statistically independent, zero-mean, random processes X(t) and ¥(¢) lave 
autocorrelation functions 
Ryx(t) = ew ld 


and 
Ryy(t) = cos (271) 
respectively. : 
ra Find the autocorrelation function of the sum W(t) = X(t) + ¥(0), : 

(b) Find the autocorrelation function of the difference M4() = X(t) - Y(t). 

(c) Find the cross-correlation function of W,() and W,(1). he 
6-18 Define a random process as X(t) = p(t + £), where p(t) is any periodic waves 
form with period T and ¢ is a random variable uniformly distributed on the inter- 
val (0, T). Show that 


sees ES 


- 
E(X()X(t + t)] = = { PPS + t) do = Ryx(t) 


eas 


ag 


‘ *6-19 Use the result of Problem 6-18 to find the autocorrelation aes of 
random processes having periodic sample function waveforms p(t) define 

(a) by Figure P6-7 with ¢ = 0 and 4t) < T, and 

(b) by Figure P6-8 with e = 0 and 2t) < T. or 
6-20 Define two random processes by X(t) = py(t 7 &) and YQ= pal! | t) wich 
p,(t) and p,(t) are both periodic waveforms with period Tand ¢ is a ran = 
able uniformly distributed on the interval (0, T). Find an expression for the cro 
correlation function ELX(t)¥(t + 1)]. 
6-21 Prove: 

(a) (6.3-4) and (b) (6.3-5). 

- ive arguments to justify (6.3-9). 
i Ge random fee having the autocorrelation function shown in 
Figure P6-23, find: 

(a) ELX(1)] 


ea 


fowiss 


SSyetk 


(b) E[X?(9] and (c) o%. 


Ryx(r) 


Figure P6-23 
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6-24 A random process Y(t) = X(t) — X(t + t) is defined in terms of a process 
X( that is at least wide-sense stationary. 
(a) Show that the mean value of Y(t) is 0 even if X(t) has a nonzero mean 


value. 
(b) Show that 
a} = 2[Ryx(0) - Ryx(t)] 


(c) If ¥(Q) = X(t) + X(t +1), find ELY(9) and a3. How do these results 
compare to those of parts (a) and (b)? 
6-25 For two zcro-mean, joinlly wide-sense stationary random processes X(t) 
and Y(t), it is known that at = 5 and a2 = 10, Explain why cach of the following 
functions cannot apply to the processes if they have no periodic components. 


(a) Ryx(t) = 6u(t) exp (—3t) (b) Ryx(t) = 5 sin (5t) 
(c) Ryy(t) = OC + dry"! (d) Ryy(t) = —cos (Gt) exp (—It]) 


sin (3t) }? in (10 
(ec) Ryy(t) = jee] (f) Ryy(t) = 6 + (ae) 


6-26 Given lwo random processes X(f) and Y(t). Find expressions for the aulo- 
correlation function of W(t) = X(Q + Y¥(q it: 
(a) X(N) and Y(t) are correlated. 
(b) They are uncorrelated. 
(c) They are uncorrelated with zero means. 
6-27 Usc (6.3-19) to prove (6.3-17). 
6-28 Let X(t) be a stationary continuous random process that is differentiable. 
Denote its time-derivative by X(t). 
(a) Show that E[X(0] = 0. 
(b) Find Ry x(t) in terms of R xx(t). 
(c) Find Rgx(t) in terms of Ry x(t). (Hints Use the definition of the derivative 
; tte) — X(t 
Xt) = lim X(t + ) = X(0 
e~0 
and assume the order of the limit and expectation operations can be inter- 
changed.) 
6-29 A gaussian randem process has an autocorrelation function 
Ryy(t) = 6 exp (—{11/2) 
Determine a covariance matrix for the random variables X(1), X(¢ 4+ 1), X(t + 2), 
and A(t -F 3). 
6-30 Work Problem 6-29 if 
sin (nt) 


Ryx(t) = 6 
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6-31 An ensemble member of a stationary random, process X(t) is sampled at N 
limes t;, = 1, 2,..., N. By treating the samples as random variables X, = X(t), 
an estimate or measurement ¥% of the mean value X = E[X()] of the process is 
sometimes formed by averaging the samples: 


4 17 
xaq Dx 


(a) Show that E[XJ = X. 
(b) If the samples are separated far enough in time so that the random vari- 
ables X, can be considered statistically independent, show that the variance of the 


estimate of the process mean is 
(og)? = 03/N 


6-32 For the random process and samples defined in Problem 6-31, let an esti- 
mate of the variance of the process be defined by 


Show that the mean value of this estimate is 


—1 
N ah 


E[ot] = 


6-33 Assume that X(t) of Problem 6-31 ig a zero-mean stationary gaussian 
process and let 


N 

+ _! 2 

oy =~ DX 

be an estimate of the variance a3 of X(t) formed from the samples. Show that the 
variance of the estimate is 


4 
: >» 26 
variance of a} = ah 


(Hint: Use the facts that E[X?] = 03, E[X*] =0, and E(X*] = 30% fora 
gaussian random variable having mean zero.) 
6-34 How many samples must be taken in Problem 6-33 if the standard devi- 
ation of the estimate of the variance of X(t) is to nol exceed 5% of a2? 
*6-35 A complex random proccss Z(t) = X(t) +49 is defined by jointly station- 
ary real processes X(t) and Y(t). Show that 
EC | Z(t) 77) = Rxx(0) + Rrvl0) 
*6-36 Let X,(0, X20, %1(9) and Y,(t) be real random processes and define 
ZQ= X(t 2.00 = X21) JY) 
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Find expressions for the cross-correlation function of Z,(t)and Z,( if: 
(a) All the real processes are correlated, oe 
(b) They are uncorrelated. 
(c) They are uncorrelated with zero means. 


*6. ray, ey efatlonar 
; 37 Tet Z(t) be a stationary complex random process with an autocorrelation 
unction R,,(t). Deline the random variable 


at+T 
W = [ Z(t) dt 


where T > 0 and a are real numbers. Show that 


i 
EUW DP] = [ie I tY)Rz2(t) de 


ADDITIONAL PROBLEMS 


ee For a random process X(t) it is known that f(x, Xo, X35 bs, tg, €3) = 
Ae, ae Xai ly tA, ty + A, ty + A) for any ¢,, t2, ty and A. Indicate which of 
‘ e Se aenenls ire unequivocably true: X(t) is (a) stationary to or- 
ee a ) stationary to order 2, (c) stationary to order 3, (d) strictly stationary 
e) wide-sense stationary, (f) not stationary in any sense, and (y) ergodic. 
ite random Process is defined by X(t) = X, + Vet where Xo and V are sta- 
oy independent random Variables uniformly distributed on intervals 
nate ee a), Saar Find (a) the mean, (b) the autocorrelation 
varia i i i 
a nce functions of X(t). (d) Is X(t) stationary in any sense? If 
ae Find the first-order density of the random process of Problem 6-39 
(6) Plot the density for 6 = k(Xo, ~ Xo,)(V¥, — V,) with k =0 %, 1, and 2. 
Assume V, = 3Y, in all plots. a 
6-41 Assume a wide-sense stationar 
\ ary process X(t) has a known mean X and 3 
known autocorrelation function Ryx{t). Now suppose the process is observed . 


time ft, and we wish to estimate, that is i 
: , predict, what the process will be at ti 
t, + t with t > 0, We assume the estimate has the form be 


R(t, += aX(t,) + B 
where a and £ are constants, 
(a) Find « and f so that the mean-squared prediction error 
er e* = EL{X(t, + 1) — R(t, + 1)}?) 
is minimum, 


(b) Find the minimum mean-squared error in terms of Ry,(z). Develop an 
alternative form in terms of the autocovariance function. 


6-42 Find the time average and time autocorrelation function of the random 


process of Example 6.2-1. Compare these r i isti 
2-1, esults with the statistic: i 
autocorrelation found in the example. en 
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6-43 Assume that an ergodic random process X(‘) has an wutocorrelation 
function 


2 
Ryg(t) = 18 + a [1 + 4 cos (12t)] 


6+ 

(a) Find |X|. 

(b) Does this process have a periodic component? 

(c) What is the average power in X(0)? 
6-44 Define a random process X(t) as follows: (1) X(f) assumes only gne of bwo 
possible levels | or —1 at any time, (2) X(t) switches back and forth between ils 
two levels randomly with time, (3) the number of level transitions in any time 
interval t is a Poisson random variable, that is, the probability of exactly & tran- 
sitions, when the average rate of transitions is A, is given by ((Ar)/kE] exp (- At), 
(4) transitions occurring in any time interval are statistically independent of tran- 
sitions in any other interval, and (5) the levels at the start of any interval are 
equally probable. X(1) is usually called the random telegraph process, WC is an 
example of a discrete random process. 

(a) Find the autocorrelation function of the process. 

(b) Find probabilities P{X(0) = 1} and P{X(t) = — 1} for any t 

(c) What is E[X())? 

(d) Discuss the stationarity of X(?). 
6-45 Work Problem 6-44 assuming the random telegraph signal has levels 0 
and |. 
6-46 X¥ =6 and Ryg(t, 6+ 1) = 36 + 25 exp (—[t]) for a random process X(1). 
Indicate which of the following statements are true based on what is known with 
certainty. X(t) (a) is first-order stationary, (b) has total average power of 6L W, (ce) 
is ergodic, (d) is wide-sense stationary, (¢) has a periodic component, and ({) has 
an ac power of 36 W. 


6-47 A zero-mean random process X(t) is ergodic, has average power of 24 W, 
and has no periodic components. Which of the following can be a valid auto- 
correlation function? If one cannot, state at least one reason why. 
(a) 16 + 18 cos (3z), (b) 24Sa?(2r), (c) CL + 37] 7! exp (—6r), and (d) 24d(t — 1). 


6-48 Use the result of Problem 6-18 to find the autocorrelation function of a 
random process with periodic sample function waveform p(t) defined by 


p(t) = A cos? (2n1/T) 


where A and T > 0 are constants, 


6-49 An engincer wants to measure the mean value of a noise signal that can be 
well-rmodeled as a sample function of a gaussian process. He uses (he sampling 
estimator of Problem 6-31. After 100 samples he wishes his estimate to be within 
+0.1 V of the truc mean with probability 0.9606. What is the largest variance the 
process can have such that his wishes will be truc? 
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6-50 Let X(t) be the suin ofa deterministic signal s(t) and a wide-sense stationary 
noise process N(1). Find the mean valuc, and autocorrelation and autocovariance 
functions of X(t). Discuss the stationarity of X(f). 


6-51 Random processes X(t) and Y(t) are defined by 
X(t) = A cos (wot + 9) 
Y(t) = B cos (mot + 9) 


where A, B, and we are constants while © is a random variable uniform on 
(0, 2n). By the procedures of Example 6.2-1 it is easy to find that X() and ¥(() are 
zero-mean, wide-sense stationary with autocorrelation functions 


Ryx(t) = (A7/2) cos (9 1) 
Ryy(t) = (B?/2) cos (wo 1) 

(a) Find the cross-correlation function Ryy(t, ¢-+ 1) and show that X(‘) and 
Y(t) are jointly wide-sense stationary. 

(b) Solve (6.4-2) and show that the response of the system of Figure 6.4-1 
equals the truc cross-correlation function plus an crror term e(T) that decreases 
as T increases. 

(c) Sketch | e(T)| versus T to show its behavior. How large must 7 be to make 
Je(T)| less (han 1% of the largest value the correct cross-correlation function can 
have? 


6-52 Consider random processes 
X(t) = A cos (wot + 9) 
Y(t) = B cos (w,t + 9) 


where A. B, w,, and (vg are constants, while © and @ are statistically independent 


random variables uniform on (0, 27). 

(a) Show that X(1) and Y(t) are jointly wide-sense stationary. 

(b) 1 © = @ show that X(t) and Y(t) are not jointly wide-sense stationary 
unless (2, = Wo- 


6-53 A zcro-mean gaussian random process has an autocorrelation function 


) Ryle =u ~(I1/6) [e156 


0 elsewhere 


Find the covariance function necessary lo specify the joint density- of random 
variables defined at times ¢; = 2(i — 1), f= 1,2, ..65 5. Give the covariance matrix 
for the X, = X(t). 


6-54 If the gaussian process of Problem 6-53 is shifted to have a constant mean 
& = ~2 but all else is unchanged, discuss how the autocorrelation function and 
covariinee matrix change. What is the effect on the joint density of the five 


random variables? 
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*6-55 Extend Example 6.6-1 to allow the sum of complex-amplitude unequal- 
frequency phasors. Let Z,,i = 1,2,...,N be N complex zero-mean, uncorrelated 
random variables with variances o3,. Form a random process 


N 
Z(t) = Yo Ze!" 


imi 


where w, are the frequencies of the phasors. 

(a) Show that E[Z(t)] = 0. 

(b) Derive the autocorrelation function and show that Z(t) is wide-sense 
stationary. 


*6-56 A complex random process is defined by 
Z(t) = exp NL 


where Q is a zero-mean random variable uniformly distributed on the interval 
from Wo ~ Aw to wy + dw, where wo and Aw are positive constants. Find: 

(a) the mean value, and (b) the autocorrelation function of Z(t). 

(c) Is Z(t) wide-sense stationary? 


*6-57 Work Problem 6-56 except assume the process 
Z(t) = e™! + e7 MM = 2 cos (Nt) 


*6-58 Let X(t) and Y(t) be statistically independent wide-sense stationary real pro- 
cesses having the same autocorrelation function R(t). Define the complex process 


Z(t) = X(t) cos (Wot) + Y(t) sin (Wo t) 


where Wp is a positive constant. Find the autocorrelation function of Z(t). Is Z(t) 
wide-sense stationary ? 


Ny 
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SPECTRAL CHARACTERISTICS OF 
RANDOM PROCESSES 


7.0 INTRODUCTION 


2 - the foregoing discussions concerning random processes have involved the 
ia omain, That is, we have characterized processes by means of autocorrela- 
‘ M eee and covariance functions without any consideration of 
say ra oe As is well known, both time domain and frequency domain 
Ay si Les hods exist for analyzing linear systems and deterministic waveforms 
: a : out random waveforms? Is there some way to describe random pro- 
ae _ he eee domain? The answer is yes, and it is the purpose of this 
apler to introduce (he most important con $ 
cepts that apply to chi erizi 
random processes in the frequency domain. Papeete 
ape — oe of a deterministic waveform is obtained by Fourier 
ansiorming the waveform, and the reader would b i i 
r ! vi e correct in concluding that 
ae Nee play an important role in the spectral chiaraciedvalton of 
a ae waveforms. However, the direct transformation approach is not altrac- 
re pe waveforms because the transform may not exist. Thus spectral 
is of random processes requi i seeninigii 
ue p quires a bit more subtlety than do deterministic 
i oy appropriate spectrum to be associated with a random process is intro- 
ope ee oe The concepts rely heavily on theory of Fourier 
s. Readers wishing to refresh their b i 
ackground on Fourier theory ¢ 

: are 

referred to Appendix D where a short review is given. ” 


172 


SPECTRAL CHARAC FERISLIUS UP INA ietanzis 1 seen teres 8 ow 
7,1 POWER DENSITY SPECPRUM AND ITS PROPERTIES 


The spectral properties of a deterministic signal x(t) are contained in ils Fourier 
transform X(w) given by 


X(w) = a x(he7l"! dt (7.1-1) 


“ww 


The function X(w), sometimes called simply the spectrum of x(t), has the unit of 
volts per hertz and describes the way in which relative signal voltage is distrib- 
uted with frequency. The Fourier transform can, therefore, be considered to be a 
voltage density spectrum applicable to x(1). Both the amplitudes and phases of the 
frequencies present in x(t) are described by X(w). For this reason, if X(cw) is 
known then x(t) can be recovered by means of the inverse Fourier transform 


die { X(w)el" deo (7.1-2) 
an Jaw, 


In other words, X(w) forms a complete description of x(t) and vice versa. 

In attempting to apply (7.1-1) to a random process, we immediately encoun- 
ter problems, The principal problem is the fact that X(w) may not exist for most 
sample functions of the process. Thus, we conclude that a spectral description of 
a random process utilizing a voltage density spectrum (Fourier transform) is not 
feasible because such a spectrum may not exist, Other problems arise if Luplace 


- transforms are considered (Cooper and McGillem, 1971, p. 132). 


On the other hand, if we turn our attention to the description of the power in 
the random process as a function of frequency, instead of voltage, it results that 
such a function does exist. We next proceed to develop this function, called the 
power density spectrumt of the random process. 


The Power Density Spectrum 
For a random process X(0), let x,(t) be defined as that portion of a sample func- 
tion x(t) that exists between — T and FT; that is 


x(t) -T<t<T 
x(t) = Ao 
sit {3 elsewhere a) 


Now so long as T is finite, x(t) will satisfy 


er 
{ [xO di < (7,1-4) 


-7 


t Many books call this function a power spectral density, We shall occasionally use also (he names 
power density of power spectrum. 
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and will have a Fourier transform (see Appendix D for conditions sufficient for 
the existence of Fourier transforms), which we denote X7(w), given by 


T T 
X(t) = | x(ne ee dt = | x(new i dt (7.1-5) 
-T a i 


The energy contained in x(f) in the interval (—7T, 1) ist 
T T 
ECT) = | x3tn) dt = | x(t) dt (7.1-6) 
-T -T 


Since x,(t) is Fourier transformable, its cnergy must also be related to X4(w) by 
Parseval's theorem. Thus, from (7.1-6) and (D-2!) of Appendix D 


T C) 
E(T) = { x(t) dt = = { |X (ew) |? dw (7.1-7) 
T 2n c) 


By dividing the expressions in (7.1-7) by 2T, we obtain the average power 
P(T) in x(t) over the interval (—T, T): 


1 7 1 (% |x (aw)? 
”T') a — 2 aust plated tas BS FE 
P(T) oT [es (1) dt on [ OT dw (7.1-8) 


At this point we observe that |X 4(w)|?/2T is a power density spectrum because 
power resulls through its integration. However, it is not the function that we seek 
for two reasons, One is the fact that (7.1-8) does not represent the power in an 
entire sample function. There remains the step of letting T become arbitrarily 
large so as to include all power in the ensemble member. The second reason is 
that (7.1-8) is only the power in one sample function and does not represent the 
process. In other words, P(T) is actually a random variable with respect to the 
random process. By taking the expected value in (7.1-8), we can obtain an 
average power Py, for the random proccss.t 

From the above discussion it is clear that we must still form the limit as 
T—» co and take the expected value of (7.1-8) to obtain a suitable power density 
spectrum for the random process. It is important that the limiting operation be 
done last (Thomas, 1969, p. 98, or Cooper and McGillem, 1971, p. 134). After 
these operations are performed, (7.1-8) can be written 


T ) 2 
Pyy = lim a { ELX2(9) dt = a { lim EEX AON] ey (7.1-9) 
- 2n aT 


T-@ 2T J- -o T~o 


¢ We assume a real process X(!) and interpret x{f) as cither the vallage across 0 1-2 impedance or 
the current (hrough 1:2. Tn other words, we shall assume a 1-2 real impedance whenever we discuss 
energy of power in subsequent work, unless specifically stated otherwise. 

f In taking the expected value we replace x(t) by XW) in (7.1-8) since the integr 
operation performed on all sample functions of X(i). 


al of x2(f) is an 
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Equation (7.1-9) establishes two important facts. First, average power Py, in 
a random process X(t) is given by the time average of its second moment: 


T 
Pyy = lim =| ELX2(t)] dt = A{ELX7(1)}} (7.1-10) 


Toy -T 


For a process that is at least wide-sense stationary, E[X7(0)] = x7, a constant, 
and Pyy = X2. Second, Pyx can be obtained by a frequency domain integration. 
If we define the power density spectrum for the random process by 


. EEX o)27) 
Syx(@) = lim > TA-V 
Oe ae va 
the applicable integral is 
{ oo 
Prxx = 57 __ fe) dw (7.1-12) 


from (7.1-9). Two examples will illustrate the above concepts. 


ee 


Example 7.1-1 Consider the random process 
X(t) = A cos (wot + O) 


where A and (uy are real constants and © is a random variable uniformly dis- 
tributed on the interval (0, 2/2). We shall find the average power Pyy in X(t) 
by use of (7.1-10). Mean-squared value is 


2 2 
E[X2(t)] = ELA? cos? (wot + ©)] = lS + - cos (2W9t + 20) | 
A A? a[2 2 
-4.4) = 98 (2m t + 20) dO 
ee ae 


ra 2a sin (2Wpo t) 


This process is not even wide-sense stationary, since the above function is 
time-dependent. The time average of the above expression is 


A a m on | : : | 
t = fi ini Nt eat: C4 
{EL (t))} meh oT ie 2 : sin (29 ¢) | dt 
which easily evaluates to 


Pax = A{ELX*(]} = 47/2 


} 
| 


pees: Ser 
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eee 
Example 7.1-2 We reconsider the Process of the above example to find 


S8xx(w) and average power Py, by use of (7.1-11) and (7.1-12), respectivel 
z First we find X,(w): ee 


T 
X,(w) = [ A cos (gt + ©) exp (—jut) de 
T 


wpm 


. 
exp (jO) | exp [i(wo — w)t] dt 
oe a 


A T 
+> exp (—jO) { exp [—j(wo + w)t] dt 
eee -T 


sin [(@ — @)T] 


= AT exp 
had aay a 
+ AT exp (—j@) SA Mw + wo) TI wT] 


(@ + Wo)T 


Next we determine | X (a) |? = X7(w)X#(w) and find its expected value. After 
some simple algebraic reduction we obtain 


EC|X{w)|?] 2 A’n t sin? [(w — w,)T] T sin? ((@ + wo} 
QT 2 ln [o-o)T? *2 fo@+rayT? | 
Now it is known that 


_ Tf sin (aT) ]? 
at ef “T ] = O(a) 


(Lathi, 1968, p. 24), so (7.1-1 1) and the above result give 
A?n 
. 8xx(@) = aC [5(w — wo) + 5(w + w)] 
Finally, we use this result to obtain average power from (7.1-12): 
co) A’n At 

Parag |Z [Ble ~ a) + Sw + w9)} deo = 
Thus, Py, found here agrees with that of the earlier Example 7.1-1, 
ee eeeeeeeeeeSeeeSFSsSssFs 

Properties of the Power Density Spectrum 


The power density spectrum possesses a number of important properties: 


ia 
{. 
t 


(1) Syx(w) 20 (7.1-13) * 
(2) 38xx(—@) = Sxx(@) X(t) real (7.1-14) 
(3) Syx(w) is real (7.1-15) 

| oo 
(4) ms [7 sexton dw = A{E[X7(t)]} (7.1-16) 


SPECTRAL CHARACTERISTICS OF RANDOM PROCESSES 177 


Property | follows from the definition (7.1-11) and the fact that the expected 
value of a nonnegative function is nonnegative. Similarly, property 3 is true from 
(7.1-11) since | X7(w) |? is real. Some reflection on the properties of Fourier trans- 
forms of real functions will verify property 2 (see Problem 7-9). Property 4 is just 
another statement of (7.1-9). 

Sometimes another property is included in a list of properties: 


(5) Sy s(t) = w7Syy(w) (7.1-17) 
It says that the power density spectrum of the derivative X(t) = dX(O/dt is a? 
limes the power spectrum of X(t). Proof of this property is left as a-reader exer- 


cise (Problem 7-10). 
A final property we list is 


(6) =~ i Sxx(w)e!! do = ALRyx(t, 6+ 1] (7.1-18) 
Syx(w) = i ALRyx(t, C+ te7 "dt (7.1-19) 


It states that the power density spectrum and the time average of the autocor- 
relation function form a Fourier transform pair. We prove this very important 
property in Section 7.2. Of course, if X(t) is at least wide-sense stationary, 
ALR xx(t, (+ 1)}] = Ryx(t), and property 6 indicates that the power spectrum and 
the autocorrelation function form a Fourier transform pair. Thus 


Syy(w) = I Ryx(e 72" dt (7.1-20) 
a o 
Rys(t) = >= [7 ssatoret deo (7.1-21) 
for a wide-sense stationary process. 


Bandwidth of the Power Density Spectrum 


Assume that X(t) is a lowpass process; that is, ils spectral components are clus- 
tered near w = 0 and have decreasing magnitudes at higher frequencies. Except 
for the fact that the area of 8y,(c) is not necessarily unity, Syy(w) has character- 
istics similar to a probability density function (it is nonnegative and real). Indeed, 
by dividing 8; ,(cw) by its area, a new function is formed with area of unity that is 
analogous to a density function. : 

Recall that standard deviation is a measure of the spread in a density func- 
tion. The analogous quantity for the normalized power spectrum is a measure of 
its spread that we call rms bandwidth,t which we denote W,,,(rad/s). Now since 
$y (cw) is an even function for a real process, its “mean value” is zero and its 


t The notation rms bandwidth stands for rout-mean-squared bandwidth. 
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“standard deviation” is the square root of its second moment. Thus, upon nor- 
matization, the rms bandwidth is given by 


n 2e 
FT eee (20 w?Syxlw) dao (7.1-22) 


ms fone Syxle) da 


Example 7.1-3 Given the power spectrum 
Se ala 10 
S$ 5 ») = 5 
xm) = FT (w/10)?)? 


where the 6-dB bandwidth is 10 radians per second, we find Wos- First, 
using (C-28) from Appendix C, 


i 10 de = 108 ts dn 
a Ll + (w/l0P o -@ (100 + 7)? 


= 105 ee ‘ gies wn'(2 a 
- FOO(100 + 0?) |— a 2000 10] \~ 


= 50x 
Next, from (C-30) of Appendix C: 


7 10? dew Ss w? dw 
_—? = = 10 —- 
_o EL (eo 10)" _, (100 + 0) 


1080 ——— +d n(2) I! } 
(100 + «?) |-o 20° (Oy lee: 


5000n 


[000% 
Wons = “son” = 10 rad/s 


are equal in (his case, they 


" 


Thos 


Although 14, and the 6-dB bandwidth of Syxl) 


are not equal i peneral. 


————oeo 


The above concept is readily extended to a process that has a bandpass form 


that is, its significant spectral components cluster near some 
If we assume that the process X(t) is real, Sxy() will be 
0. With this assumption we define a 


of power spectrum, 
frequencies (9 and —@- 
real and have even symmetry about @ = 
mean frequency @o by 


@ aSyy(@) dn (7.1-23) 


Dy = 
On fy Syl) day 
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and rms bandwidth by 


W2,, = 4 fo (co — dig) Sy x(a) dao (7.1-24) 


fa Syx(o) deo 


‘The reader is encouraged to sketch a few lowpass and bandpass power spectrums 
and justify for himself why the factor of 4 appears in (7.1-24). 


72 RELATIONSHIP BETWEEN POWER SPECTRUM 
AND AUTOCORRELATION FUNCTION 


In Section 7.1 it was stated that the inverse Fourier transform of the power 
density spectrum is the time average of the autocorrelation function; that is 


i om 
= [ Syylcaelet dan = ALR y(t. 6+ t)) (7.2-1) 
This expression will now be proved, 

If we use (7.1-5), which is the definition of X;(«), in the defining equation 
(7.1-11) for the power spectrum we havet 


: Pare oe Fare T 
Sxx(M) = aim As | X(t,el' dty { i a a,| 


a -T 


. { T T 
= fim val | ELX(t,)X(tq)]e "27 dey dey (7.2-2) 
-TJ-T 


Tam 2 ws 


The expectation in the integrand of (7.2-2) is identified as the autocorrelation 


function of X(t): 


EEX (ty )X(a)] = Ryvltae 2) -T <(t, and t,)<T (7.2-3) 
Thus, (7.2-2) becomes 
: . 1 T Tr 
Sxa(e) = tien am i [Raat ten FeO) dty dt (7.2-4) 
Suppose we next make the variable changes 
t=, dt srt, (7.2-Sa) 
tel,-=Hhh-! dt = dt, (7.2-5b) 
in (7.2-4); we obtain 
; prs es 
Syx(w) = meee T fe ec t+) dee" dt (7.2-6) 


t We use X(1) in (7.1-5), rather than x(1), to imply that the operations performed take place on (ne 
process. as Opposed to one sumple function, 


j 
: 
+f 
i 
H 


ods 
? 
5. 
, 
in 
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Next, taking the limit with respect to thé t integral first will allow us to inter- 


change the limit and t integral operations to get 


w T 
Syx(v) = i { lim vale Ryxlt, t +1) athe-™ dt (7.2-7) 
T7 0 2T 


The quantity within braces is recognized as the time average of the process auto- 
correlation function 


I T 
ACR yx(t, 6+ 1)] = lim — _ Raat t+r)dt (7.2-8) 
T-2 2T 
Thus, (7.2-7) becomes 
Syy(to) = | ALRyxlt, 6+ tie de (7.2-9) 


which shows that Syy(w) and A[Ryx(t, ¢ + t)] form a Fourier transform puir: 
ALR x x(t, ¢ + t)] 4 8yx(w) (7.2-10) 


This expression implies (7.2-1), which we started out to prove. 
For the important case where X(t) is at least wide-sense plaonaty: 
ALR xx(t, 6+ 1)] = Ryxx(t)} and we get 


§xx(@) = | Ryx(t)e7/** dt (7.2-11) 
"~& 
| a 
Rxx(t) = on | Sxx(we!”* dw (7.2-12) 
or 
Ry x(t) 4 8xx(e) (7.2-13) 


The expressions (7.2-11) and (7.2-12) are usually called the Wiener-Khinchin rela- 
tlons after the great American mathematician Norbert Wiener (1894-1964) and 
the German mathematician A. I. Khinchin (1894-1959). They form the basic link 
between the time domain description (correlation functions) of processes and 
their description in the frequency domain (power spectrum). 

From (7.2-13), it is clear that knowledge of the power spectrum of a process 
allows complete recovery of the autocorrelation function when X(t) is at least 
wide-sense stationary; for a nonstationary process, only the time average of the 
autocorrelation function is recoverable from (7.2-10). 


Example 7.2-1 The power spectrum will be found for the random process of 
Example 6.2-1 that has the autocorrelation function 


Ryx(t) = (47/2) cos (wo 1) 
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where A and wo are constants. This equation can be written in the form 


At 
Ryx(e) = (elt + e700 


Now we note that the inverse transform of a frequency domain impulse func- 
tion is 
1 


oe [7 see dw =— 


from (A-2) of Appendix A. Thus : 
] ++ 2nd(w) 

and, from the frequency-shifling property of Fourier transforms given by 

(D-7) of Appendix D, we get 


OB ANd — Wy) 


. 


By using this last result, the Fourier transform of Ryx{t) becomes 
§xx(@) = 2 [5(w — Wo) + 5(w + Wo)] 


This function and Ryx(t) are illustrated in Figure 7.2-1. 


Figure 7.2-1 The autocorrelation 
function (a) and power density 
spectrum (b) of the wide-sense 
stationary random process of 
(b) Zxample 7.2-1, 
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7.3 CROSS-POWER DENSITY SPECTRUM 
AND ITS PROPERTIES 


Consider a real random process W(t) given by the sum of two other real pro- 
cesses X(t) and ¥(1): 


W(t) = X(t) + ¥() (7.3-1) 
The autocorrelation function of W(t) is 
Ryaylty tt) = ELW(QW(t + 9) 
= (LX) + YOMEXG +) + YE + 90) 
= Ryxlt, t+ t) + Ryylt, t+ t) 
+ Ryylt, C+ t+ Ryxl 6+ t) (7.3-2) 
Now if we tike the lime average of both sides of (7.3-2) and Fourier transform 
the resulting expression by applying (7.2-9), we have 
Syyr(v) = Sy yen) bf Syy(eo) + FLALR yt, OF t)]} + F(ALR yx, + t)}} (73-3) 


where F {+} represents the Fourier transform. It is clear that the left side of (7.3-3) 
is just the power spectrum of W(t). Similarly, the first two right-side terms are the 
power spectrums of X(t) and Y(¢), respectively. The second (wo right-side terms 
are new quantities that are the subjects of this section. It will be shown that they 
are cross-power density spectrums defined by (7.3-12) and (7.3-14) below. 


The Cross-Power Density Spectrum 


For two real random processes X(t) and Y(t), we define x,(t) and y,(f) as trun- 
cated ensemble members; that is ! 


x(t) —-T<t<T 
ot) = 7.3-4 
alt i elsewhere ee 
and 
y(t) -T<1<T 73-5) 
m= elsewhere ee 


Both xp) and y7(e) are assumed to be magnitude integrable over the interval 
(-T, T) as indicated by (7.1-4). As a consequence, they will possess Fourier 


transforms that we denote by X7(w) and Y,{w), respectively: 
x(t) X yf) (7.3-6) 


yf) Sad Y,(«) (7.3-7) 
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We next define the cross power Py,(T) in the two processes within the inter- 
val (—T, T) by 


1 (7 T 
P,,(T) = 7 { rds dt = = | x() yo dt (7.3-8) 
= -T 


Since Xr{!) and yx{t) arc Fourier transformable, Parseval's theorem (D-20) 
applies; its left side is the same as (7.3-8). ‘Thus, we may write 


i 7 ie ecu, 2 
Py) = 5 { x00 dt = 5, ig “rane deo (7.3-9) 


This cross power is a random quantity since its value will vary depending on 
which ensemble member is considered. We form the average cross powcr, 
denoted Py,(T), by taking the expected valuc in (7.3-9). The result is 


- bad * 4 
Pr(Ti=ge [Rat nde= ge [” Seber 
-T —— 


Finally, we form the total average cross power Pyy by letting T— co: 


dw — (7.3-10) 


© 


: 1 T | E(X* : 
Pyy = lim — | Ryylt, 2) dt == : [X Hw) ¥{o)] 
ee eet 1 5 ie wl dtm oe | lim SE deo (7.5411) 


It is clear that the integrand involving w can be defined as a cross-power density 
spectrum; it is a function of w which we denote 


E[XH@) ¥{o)] 


Syy(w) = lim = 
xy(@) Ati oT (7.3-12) 
Thus 
P a 
xy 90 | Sxrl) dw (7.3-13) 


By repealing the above procedure, we can also define another cross-power 
density spectrum by 


. ELY#(w)X 
Syx(@) = lim ELY HO) rel Hee Hel] (7.3-14) 
Cross power is given by 
1 on 
Py, = on [ Syx(w) dw = Pyy (7.3-15) 


Total cross power Pyy + Py, can be interpreted as the additional power two pro- 
cesses are capable of generating, over and above their individual powers, due to 
the fact that they are correlated. 
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Properties of the Cross-Power Density Spectrum 


Some properties of the cross-power spectrum of real random processes X(t) and 
Y(t) are listed below without formal proofs. 


(1) Syy(@) = 8yx(—@) = 8F,(a) (7.3-16) 

(2) Re [Syy(w)] and Re [Sy,(w)] are even functions of w (see Problem 7-40). 

(7.3-17) 

(3) Im [Syy(w)] and Im [8y,(w)] are odd functions of w (see Problem 7-40). 

(7.3-18) 

(4) Syy(w) = O and Syy(w) = 0 if X(1) and Y(t) are orthogonal. (7.3-19) 
(5) If X(t) and Y(t) are uncorrelated and have constant means ¥ and Y 

Sxy(w) = Syx(w) = 2X PS(w) . (7.3-20) 

(6) ALRyylt, ¢ + 1)] + Syy(o) (7.3-21) 

. ALRyglt, t + t)] + Syx(w) (7.3-22) 


In the above properties, Re [-] und Im [+] represent the real and imaginary 
parts, respectively, and A[+] represents the time average, as usual, defined by 
(6.2-21). Hers 

Property 1 follows from (7.3-12) and (7.3-14). Properties 2 and 3 are proved 
by considering the symmetry that X,(w) and Y,{w) must possess for real pro- 
cesses. Properties 4 and 5 may be proved by substituting the integral (Fourier 
transform) forms for X,(w) and Y,({w) into E[X#{w)¥;(w)] and showing that the 
function has the necessary behavior under the stated assumptions, 

Property 6 states that the cross-power density spectrum and the time average 
of the cross-correlation function are a Fourier transform pair; its development is 
given in Section 7.4. For the case of jointly wide-sense stationary processes, 
(7.3-21) and (7.3-22) reduce to the especially useful forms 


8xy(w) = " Ryy(t)e J" dt (7.3-23) 
Syx(w) = [ Ryx(t)e7 2"! de (7.3-24) 
Ryy(t) = = [i serwret dw ; (7.3-25) 
Ryx(t) = = [7 seroret dw (7.3-26) 


Example 7.3-1 Suppose we are given a cross-power spectrum defined by 


abe a+ jbw/W -W<a<W 
sad sean elsewhere 
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where W > 0, a and 6 are real constants. We use (7,3-25) to find the cross- 
correlation function. It is 


,ry bw Jor of 
Ryerlt) = > le a+ er? dap 


Ww 


oe Jee day 4 ee cwel™® lw 
=— e aa J anW 


-W -W 
On using (C-45) and (C-46) this expression will readily reduce to 


a ejut i 5 b Jor ad “3 } 
Ryy(t) = 57 |= si ne 2nW ‘e | Git)? yew) ° 


2n |, jt 
= ge ((aWr — b) sin (Wt) + bWt cos (IV1)] 
nWr? 


ence 


«7.4 RELATIONSHIP BETWEEN CROSS-POWER SPECTRUM 
AND CROSS-CORRELATION FUNCTION 


In the following discussion we show that 


00 ; | rT aos) 2 
Syy(w) = [: {im oT | Ravlts t+) art Jor dt (7.4+1) 


-o (T-0 -7 
as indicated in (7.3-21). 
The development consists of using the transforms of the truncated ensemble 
members, given by s 
X7(w) = | x(ne~e" dt (7.4-2) 


-T 
r 
Y,{w) = | ye" de (7.4-3) 
os 
in (7.3-12) and then taking the expected value and limit as indicated (o obtain 
Syy(w). From (7.4-2) and (7.4-3): 
“ 
X Fw) Yi, (ce) = | xe jel! dt | sie Wide, 
‘: 


-' 


pe RE 
= | | x(ty)y(tqe 2 dt, dt (7.4-4) 
T 


-TJ-T 
Now by changing variables according to (7.2-5), dividing by 2T, and taking the 
expected value, (7.4-4) becomes 


ELX Hw) Velo] = Al is [’ x(thy(t +t) arfe-m ac| 
2T T 


T- T 
on | ‘sm | Ryy(t, ¢ +t) infer dt (7.4-5) 


-T 
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After the limit is taken: 

ELX Ham) Yeo 

Syy(o) = lim ELX Ho) Voy) 
Tm 


[Fela oan 
— | Ryylt, Ct) de pe 2" dt 
Toa J-Tot 27 -T 
> rot 
= tim me | Rav C+ cde pe" de 
‘gitpam al Jey 


which is the same as (7.4-1). Sinec (7.4-0) is a Fourier transform, and such (rans- 
forms are unique, the inverse transform applics: 


WW 


(7.4-6) 


Tr 1c” 
lim a { Ryylt, C+ 1) dt = = { Sxy(w)el! deo (7.4-7) 


pow 27 Jor 
It should be noted from (7.4-7) that, given the cross-power spectrum, the 
cross-corrclation function cannot in general be recovered, only its time average 
can. For jointly wide-sense stationary processes, however, the cross-corrclation 
function Ryy(t) can be found from Syy(an) since its time average is just R yy(t). 
Although we shall not give the proof, a development similar to the above 
shows that (7.3-22) is true. 


“om 


Example 7.4-1 Let the cross-correlation function of two processes X(t) and 
Y(t) be 


Ryy(, 6-4 t) = “ (sin (@ T) + COS {co_(2t + 1))} 


where A, Band mg are constants. We find the cross-power spectrum by usc 
of (7.4-1). First, the time average is formed 


ae 
lim <= Ryy(t, C+ 1) de 
yon 2T | ro 


AB AB =.2. OES 
= —— si = = 2t + t)) dt 
255 sin (Wot) + ihn. oT <= [wol yj 


The integral is readily evaluated and is found to be zero. Finally we Fouricr 
transform the time-averaged cross-correlation function with the aid of pair 


{2 of Appendix E: 
{= sin (9 of 


~jnAB 
; 


Sxyl) = F 


tl 


il 


[5(er — ey) S(ad -F ag) ] 


4 


Dede) Pe ee —s 
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75 SOME NOISE DEFINITIONS AND OTHER TOPICS 


In many practical problems it is helpful to sometimes characterize noise (hrough 
its power density spectrum. Indeed, in the following discussions we define two 
forms of noise on the basis of their power spectrums. We also consider the 
response of a product device when one of its input waveforms is a random signal 
or noise. 


White and Colored Noise 


A sample function n(t) of a wide-sense stationary noise random process N() is 
called white noise if the power density spectrum of N(!) is a constant at all fre- 
quencies. Thus, we define 


Swn(@) = "9/2 (7.5-1) 


- for white noise, where \“p is a real positive constant. By inverse Fourier trans- 


formation of (7.5-1), the autocorrelation function of N(t) is found to be 


Rywlt) = (49/2) 6(t) 


‘The above two functions are illustrated in Figure 7.5-1. While noise derives its 
name by analogy with “white” light, which contains all visible light frequencies 
in its spectrum. 

White noise is unrealizable as can be seen by the fact that it possesses infinite 
average power: 


(7.5-2) 


i ao 
—_ { Syylo) dea = 00 (7.5-3) 


2n 


However, one type of real-world noise closcly approximates white noise. Thermal 
noixe generated by thermal agitation of electrons in any electrical conductor has a 
power spectrum that is constant up to very high frequencics and then decreases. 
For example, a resistor at temperature T in kelvin produces a noise voltage 


Ryw(t) Sw(w) 

MN Nf? 
0 bd 0 w 
(a) (n) 


Figure 7.5-1 (a) The autocorrelation function and (h) the power density spectrum of white noise. 
| Adupted fram Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book Program.) 
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‘ 


across its open-circuited terminals having the power spectrumf (Carlson, 1975, 
p. 118) 


Sya(w) = Cromclel/t) (7.5-4) 


where a = 7.64(107'?) kelvin-seconds is a constant. At a temperature of T = 
290 K (usually called room temperature although it corresponds to a rather cool 
room at 63°F), this function remains above 0.9 (%)/2) for frequencies up to 
10'? Hz or 1000 GHz. Thus, thermal noise has a nearly flat spectrum at all fre- 
quencies that are likely to ever be used in radio, microwave, or millimeter-wave 
systems.t 

Noise having a nonzero and constant power spectrum over a finite frequency 
band and zero everywhere else is called band-limited white noise. Figure 7,5-2a 


t The unit of 8,,(u) is actually volts squured per hertz. According to our convention, we obtain 
walls per hertz by presuming the voltage exists across a 1-2 resistor. 

} This stutement must be reexamined for T < 290 K, such as in some superconducting systems or 
other low-temperature devices (masers). 


Sanlw) 


-2n on x 2n Figure 7.5-2 Power density spectrum 
W Ww W W {a) and autocorrelation function (b) 
(0) of lowpass band-limited white noise. 


SPECTRAL CHARACTERISTICS OF RANDOM PROCESSES 10 


depicts such a power spectrum that is lowpass. Here 


sis -W<w<W 
Synl@) = 4 V (7.5-5) 
0 elsewhere 


Inverse transformation of (7.5-5) gives the autocorrelation function shown in 


Figure 7.5-26: : 
sin (Wt) 


Wt 


The constant P equals the power in the noise. ee 
Band-limited white noise can also be bandpass as illustrated in Figure 7.5-3. 


Ryw(t) = P (7.5-6) 


Syl) 


(eS ecclcoa 


Figure 7.5-3 Power density spectrum (a) and autocorrelation function (b) for bandpass band-limited 
white noise. [Adapted from Peebles (1976) with permission of publishers Addison-Wesley, Advanced 
Book Program.) 
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The applicable power spectrum and autocorrelation function are: 
Pr{W (Wy — (W¥/2) < |@] < Mo + (W/2) 


| - 75-7 
Sxl) . elsewhere 


and 
, sin (iW/1/2) 
(W7/2) 
where wo and $V are constants and P is the power in the noise. 
Again, by analogy with colored light that has only a portion of the visible 
light frequencies in ils spectrum, we deline colored noise as any noise that is not 
white. An example serves to illustrate colored noise. 


Ryalt) = | cos (Wy T) (7.5-8) 


Example 7.5-1 A wide-sense stationary noise process N(t) has an autocorrel- 
ation function ; 
Ryy(t) = Pew St 


where P is a constant. We find its power spectrum. It is 


a 
Spal) = | Pew Mle" dt 


-@ 


) 0 
=P | 278 +o de + P | eld ~ Jere dt 
0 


-o 


Figure 7.5-4 The autocorrelation 
function (a) and power spectrum 
(b) of the colored noise of Example 
7.51. [Adapted from Pechles (1976) 


Addison-Wesley, Advanced Book 
(i) Program.) . 


with permission of publishers, 
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These integrals casily evaluate using (C-45) to give 
P 6P 


This power spectrum is sketched in Figure 7.5-4 along with the preceding 
autocorrelation function. 


i 


Product Device Response to a Random Signal 


Product devices are frequently encountered in electrical systems. Often they 
involve the product of a random waveform X(t) (either signal or noise or the sum 
of signal and noise) with a cosine (or sine) “carrier” wave as illustrated in 
Figuie 7.5-5. The response is the new process 


Y(t) = X(t)Ag Cos (9!) (7.5-9) 


where Ag and wy are constants, We seek to find the power spectrum Syy{«) of 
Y() in terms of the power spectrum Syx(w) of X(0). 
The autocorrelation function of Y(t) is 


Ryy(t, t +t) = ELV()V(t + t)} 


= ELA2 X()X(t + 1) cos (wot) Cos (Wf + %o *)] 
Az 
= a Ryxlt, + t)[COS (wo t) + COs (2w9t + Wot)} (7.5-10) 


Even if X(t) is wide-sense stationary Y(t) is not since Ryy(t, ¢ + 1) depends on ¢. 


‘Thus, we apply (7.1-19) to obtain Syy(w) after we take the time average of 


Ryylt, ¢ + 1). Let X(t) be assumed wide-sense stationary. Then (7.5-10) becomes 
ALR yy(t, (+ 1) = 43 Ryx(t) COS (Wo 1) (7.5-11) 
On Fourier transforming (7.5-11) we have 
-Syy(0) = a3 [Sxx(@ — Wo) + Sxx(H + Wo)] (7.5-12) 
A possible power density spectrum of X(t) and that given by (7.5-12) are illus- 


trated in Figure 7.5-6. It presumes that X(r) is a lowpass process, although this is 
not a constraint in applying (7.5-12). 


X(t) Y(t) 
Product 5 
8 xx(w) Syy(w) 
Figure 7.5-5 A product of interest in electrical systems. 
[Adapted from Peebles (1976) with permission of publishers 
Ag COs (Wolf) Addison-Wesley, Advanced Book Program.] “ 
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Srxlw) 
Syx(0) -— 


Figure 7.5-6 Power density spectrums upplicuble to Figure 7.5-5; (a) at the input and (b) at the 


output. [Adapted from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book 
Program] 


Example 7.5-2 One important use of the product device is in recovery 
(demodulation) of the information signal (music, specch, etc.) conveyed in the 
wave transmitled from a conventional broadcast radio station thal uses AM 
(amplitude modulation). The wave received by a receiver tuned (o a station 
with frequency w 9/27 is one input to the product device. The other is a “local 
oscillator” signal Ay cos (wo!) generated within the receiver. The product 
device output passes through’a lowpass filter which has as its output the 
desired information signal. Unfortunately, this signal also contains noise 
because noise is also present at the input to the product device; the input 
noise is added to the received radio wave. We shall calculate the power in the 
output noise of the product demodulator. 

Let the power spectrum of the input noise, denoted X(t), be approx- 
imated by an idealized (rectangular) function with bandwidth War centered 
at + ,. Thus, 


NM 9/2 — Wy —(Wee/2) < w < —Wq + (Wex/2) 
Syxx(w) = { AM 9/2 (Wg — (Wep/2) < © < Wo + (Wey/2) 
0 elsewhere 


where A9/2 is the power density within the noise band. By applying (7.5-12) 
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the power density spectrum of the output noise Y(t) of the product device is 
readily found (by sketch) lo be 


MW AX8 = Wg — (Weat/2) <0 < = 2g + (War?) 
My Aald —Wyp/2 <0 < Wep/? 

Syl) =) ir 42/8 2a — (Wrp/2) < W < 2 + (Wre/2) 
0 elsewhere 


Now only the noise in the band —We;/2 <w < War/2 cannot be removed 
by a lowpass filter (which usually follows the product device. to remove 
unwanted noise and other undesired outputs) because the desired signal is z 
the same band. This remaining component of Syy(w) gives rise to the fina 
output noise power, denoted N,, 


7 “ 24, 
as =e is MOAB ty 2 Moda War 
tu 


-Warl2 8x 
i 


*7.6 POWER SPECTRUMS OF COMPLEX PROCESSES 


Power spectrums may readily be defined for complex processes, ae oe ei 
those processes that are at least wide-sense stationary. In terms of t eat oco iS 
lation function R,,(t) of a complex random process Z(t), the power density spec- 
trum is defined as its Fourier transform 


LJ 
Syz(w) = { Rypltye I" dt (7.6-1) 
The inverse transform applics, so 
y 7 ¥ wt Fe 
Ry2(t) = aa i _Bazleode! da (7.6-2) 


For two jointly wide-sense stationary complex processes Z,,At) and Z(t) their 
cross-power density spectrum and cross-correlation function are a Fourier trans- 


form pair: 


o 


8z,2,() = | 


—-@ 


Rzz,(t)e 7" dt (7.6-3) 


| © we = 
Ry2 (0) = ae { 8z,2,(w)el* dw (7.6-4) 


An equivalent statement is: 


RzzAt) > Szan,w) (7.6-5) 


foun 7 


afta, o 


pao | 
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aT 
Example 7.6-1 We reconsider the complex process V(t) of Example 6.6-1 and 
find its power spectrum, From the previous example 


N 
Ryv(t) = el") AD 


azt 
On Fourier transforming this autocorrelation function we obtain 


N 
Syr(w) = 5 en Y ah 


N 
y ABS (el) 


and 


N 
Ando — Wo) A 


asd 


after using pair 9 of Appendix E. 
I 


PROBLEMS 


J-1 Weare given the random process 
X(t) = A cos (Wo t + O) 


where A and wa are constants and © is a random variable uniformly distributed 


on the interval (0, 7). 
(a) Is X(t) wide-sense stationary? 
(b) Find the power in X(0) by using (7.1-10). 
(c) Find the power spectrum of X(t) by using (71-11) and calculate power 


from (7.1-12). Do your two powers agree? 
7-2 Work Problem 7-1 if the process is defined by 


X(t) = u(t)A cos (Wot + ©) 


where u(t) is the unit-step function. 
*7-3 Work Problem 7-2 assuming © is uniform on the interval (0, 2/2). 
3-4 Work Problem 7-1 if the random process is given by X(t) = A sin (cag f + ©). 


*7.5 Work Problem 7-1 if the random process is 
X(t) = A? cos? (wot + ©) 
7-6 Let A and B be random variables. We form the random proccss 
X(t) = A cos (M9 8) + B sin (a9 t) 


where an, ts a real constant 
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(a) Show that if A and B are uncorrelated with zero means and equal 
variances, then X(t) is wide-sense stationary. 

(b) Find the autocorrelation function of X(V). 

(c) Find the power density spectrum. 


7-7 A limiting form for the impulse function was given in Example 7.1-2. Give 
arguments to show that the following are also true: 


(a) lim T exp [—na?T7] = d(a) 


Te0 


(>) lim = exp [—ja| T} = d(«) 


Tam 
7-8 Work Problem 7-7 for the following cases: 
; . T sin (aT 
(a) lim Taine?) 

n aT 


T~o@ 


= 6(a) 


(b> lim TLt—lalT] = 5(c) 
T70 
{a} <1/T 
7-9 Show that (7.1-14) is true. 
7-10 Prove (7.1-17). (Hint: Use (D-6) of Appendix D and the definition of the 
derivative.) 


7-11 A random process is defined by 
Y(t) = X(t) cos (wot + ©) 


where X(t) is a lowpass wide-sense stationary process, Wo is a real constant, and 
© is 2 random variable uniformly distributed on the interval (0, 27). Find and 


* + sketch the power density spectrum of Y(t) in terms of that of X(t). Assume © is 


independent of X(1). 


7-12 Determine which of the following functions can and cannot be valid power 


J density spectrums, For those that are not, explain why. 


(a) (b) exp [—(w — 1)7] 


w® + 3w? +3 
a)? y* 

fps i) ———— 

() w* +1 a 1 +0? + jo® 


7-13 Work Problem 7-12 for the following functions. 


cos (3) 
(a) 1+? 0) (1 +?) 
el oe 
: 1 +20 + w? Si — 30" 


) 


er eer aa 


i 


2 
eee ie ee 


aren 


isexst 


we 


Eine 


8 


é 
t 
i 
4 
ah 


wis 


Sete oe 


a 


196 PROBABILITY, RANDOM VARIABLES, AND) RANDOM SIGNAL PRINCIPLES 


7-14 Given that X(Q = YM, a XA where {a,} is a set of real constants and the 
processes X(t) are stationary and orthogonal, show that 


N 
oy a} 8x,x(w) 


ball 


Sxx(w) = 


7-15 A random process is given by 
X(t) = A cos (Qt + O) 
where A is a real constant, Q is a random variable with density function /p(-), 


and © is a random variable uniformlyd istributed on the interval (0, 2) indepen- 
dent of Q. Show that the power ssa of X(t) is 


Syx(w) = TA teal + So(-)] 


7-16 If X(t) is a stationary process, find the power spectrum of 
Y(t) = A+ BX() 
in terms of the power spectrum of X(¢) if and B are real constants. 
7-17 Find the power density spectrum of the random process for which : 
Ryx(t) = P cos* (wot) 


if P and wo are constants. Determine the power in the process by use ont: 1-12), 
7-18 A random process hus the power density spectrum 
6w? 
$xx(@) = > 
xl ) I + w* 
Find the average power in the process. 
7-19 Work Problem 7-18 for (he power spectrum 
6a)? - 
(l+w?} 
7-20 Work Problem 7-18 for the power spectrum 


Syxxlw) = 


, 6c? 
5xx(w) = (1+ 0) 


7-21 Assume X(t) is a wide-sense stationary process with nonzero mean value 
X #0, Show that 


Sxx(w) = 2nX?75(w) + | Cyylt)e 2" dr 


~@ 
where Cy,x(t) is the autocovariance function of X(2). 
7-22 Fora random process X(t), assume that 


Ryy(t) = Pe“ Pe 


am 
4 
rf 
. 
{. 
‘B 
ai 
ff 
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where P > O and a > 0 are constants, Find the power density spectrum of V4). 
[Hint: Use Appendix E to evaluate the Fourier transform of R yx(t).] 
7-23 A random process has an autocorrelation function 


PCL — (21/T)] O<1s 7/2 
Ryx(t) = { PEL + 21/T)] -T/2<1<0 
0 tT<~—T/2 and t> T/2 


Find and sketch its power density spectrum. (Hint: Use Appendix E.) . 

*7.24 A random process X(t) has a periodic autocorrelation function where the 
function of Problem 7-23 forms the central period of duration 7. Find and sketch 
the power spectrum. 

7-25 If the random processes of Problem 7-14 are stationary, zero-mean, sta- 
listically independent processes, show that the power spectrum of the sum is the 
same as for orthogonal processes. For stationary independent processes with 
nonzero means, what is Sy x(w)? 

7-26 Given that a process X(t) has the autocorrelation function 


Ryy(t) = 


where A > 0, > 0, and wy are real constants, find the power spectrum of V(1). 


7-27 A random process X(t) having the power spectrum of Problem 7-19 is 
applied to an ideal differentiator. 

(a) Find the power spectrum of the differentiator’s output. 

(b) What is the power in the derivative? 
7-28 Work Problem 7-27 for the power spectrum of Problem 7-20, 
7-29 A wide-sense stationary random process X(t) is used to define another 
process by 


Aen ll cos (9 1) 


Y(N= | NE)X (Eg) de 


where h(t) is some real function having a Fourier transform H(w). Show that the 
power spectrum of Y(t) is given by 


Syylw) = Syx()| H(w)|? 


7-30 A deterministic signal A cos (gt), where A and jy are real constants, is 
added to a noise process N(t) for which 
Ww? 


Sal) = rae 


and W > Ois a constant. 

(a) Find the ratio of average signal power to average noise power. 

(b) What value of W maximizes the signal-to-noise ratio? What is the conse- 
quence of choosing this valuc of W? , 
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97-31 Find the rms bandwidth of the power spectrum 


P 
So. jopeKw 
Syx(o) = {1 + (o/WV) 
Q fol > KW 


where P, Wand K are real positive constants. If K — oo, what happens? 
7-32 Find the rms bandwidth of the power spectrum 


" _ P cos (n/2W) jal <Ww 
Sex) = Yo jwl>W 


where W > Oand P > Oare constants. 
9-33 Determine the rms bandwidth of the power spectrums given by: 


(a) Syx(to) = ‘° jj <W 
ONY = 0 he K se 

: o PUL =—jo/W¥ 1) lols W 
(b) Syx(eo) = {p wis 


where P and JY are real positive constants. 
*7.44 Given the power spectrum 


> P 


: i 
Sex) = —7 a tT (wey 
GAT [-Gry] 


where P, a, and W are real positive constants, find the mean frequency and rms 


bandwidth. 
9-35 Show that the rms bandwidth of the power spectrum of a real 


process (1) is given by 


bandpass 


W2,, = 4(W? — wo) 


where (vg is given by (7,1-23) and W? is given by the right side of (7.1-22). 
*7.36 Jointly wide-sense stationary random processes X(t) and ¥(‘) define a 
process I(t) by 
W(t) = X(0) cos (wo!) + Y(t) sin (Wo!) 
where (is a real positive constant. 

(a) Develop some conditions on the mean values and correlation functions of 


X(gQhand YC) such that §/(0) is wide-scuse stationary. 
(b) With the conditions of part'(«) applied to (0), find its power spectrum in 


terms of power spectrums of X(t) and Y(t). 
(ce) If NCA) and Y(0) are also uncorrelatec 


wn? 


{, what is the power spectrum of 


> mat ilehigS ne tee er emcees! 
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7-37 A random process is given by 
W(t) = AX() + BY(1) 


where A and B are real constants and X(t) and Y(0) are jointly wide-sense station- 


ary processes, 
(a) Find the power spectrum Swi) of W(0. 
(b) Find Swi) if X(t) and Y(t) are uncorrelated. 
(c) Find the cross-power spectrums Sxw{w) and Syip(«). 


*7.38 Define two random processes by 


X(t) = A cos (wot + ©) 


Y(t) = W(0) cos (wot + 9) 4 
where A and Wo are real positive constants, © is a random variable independent 
of W(t), and W(t) is a random process with a constant mean value W. By using : 
(7.3-12), show that % 
a 
AWn Ee 
Sxy() = 3 [d(w — a9) + d(o + Wo)] a 
regardless of the form of the probability density function of ©. 


*7.39 Again consider the random processes of Problem 7-38. 
(a) Use (6.3-11) to show that the cross-correlation function is given by 


AW 
Ry 4 1) = > {cos (‘v9 t) + E{cos (20)] cos (209! + Mot) 
— E[sin (20)] sin (2w9t + Wo 1)} 


where the expectation is with respect to © only. 

(b) Find the time average of Ryy(t, +t) and determine the cross-power 
density spectrum 5yy(w). 
7-40 Decompose: the cross-power spectrums into real and imaginary parts 
according to 


Sxy(@) = Rxy(o) + jlyy(@) 
Syx(w) = Ryx() + jlyx(w) 


and prove that 
Ryy(w) = Ry;(—@) = Ryx(o) | 
Tyy(w) = Tyy(-@) = —1yx(v) ‘ 
4-41 From the results of Problem 7-40, prove (7.3-16). 
_ 7-42 Show that (7.3-19) and (7.3-20) are (rue. 


pose See re 
Saco tae 
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7-43 (a) Sketch the power spectrum of (7,5-4) as a function of aw/T. 

(b) For what values of w will S,y,(w) remain above 0.5(4%,/2) when 
T = 42K (the value of liquid helium at one atmosphere of pressure)? These 
values form the region where thermal noise is approximately white in some 
amplifiers operated at very low temperatures, such as a maser, 
7-44 For the power spectrum given in Figure 7.5-2a, show that (7.5-6) defines the 
corresponding band-limited noise autocorrelation function. . 
7-45 Show that (7,.5-8) gives the autocorrelation function of the bandpass band- 
limited noise defined by Figure 7.5-3a. 
7-46 A lowpass random process X(t) has a continuous power spectrum 8, ,(w) 
and Syx(0) #0. Find the bandwidth W of a lowpass band-limited white-noise 
power spectrum having a density $,,(0) and the same (otal power as in X(t). 
7-47 Work Problem 7-46 for a bandpass process assuming Sy y(w ) # 0, where 
Wo is some convenient frequency about which the spectral components of X(t) 
cluster, 


*7-48 A complex random process is given by 
Z(t) = Ae 


where Q is a random variable with probability density function f,(-) and A is a 
complex constant. Show that the power spectrum of Z(t) is 


Szz(w) = An] A Pfy(eo) 


ADDITIONAL PROBLEMS 


7-49 The autocorrelation function of a random process X(t) is 
Ryx(t) = 3 + 2 exp (—41?) 


(a) Find the power spectrum of X(t). 

(b) What is the average power in X(t)? 

(c) What fraction of the power lies in the frequency band — if2 s 
wos 1/./22 
7-50 State whether or not each of the following functions can be a valid power 
density spectrum. For those (hat cannot, explain why. 


|w| exp (— 4c?) 


— 2 ; 
the (b) cos (3w) exp (—a? + j2w) 


(«) 


w® 


(12 + «?)é (d) 6 tan [12w/(1 + w?)) 


(c) 
(Sf) (—seoioo)(3 = jeo)?*(3 + yoo)? 


7-51 If Syx(w) is a valid power spectrum of a random process X(t), discuss 
whether the functions dSy y(w)/dw and d?8yy(w)/deo? can be valid power spec- 
trums. 


(e) cos? (w) exp ( —8w?) 


moon sy 


Rete 


a 


3 
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7-52 (a) Rework Problem 7-15 and show that even if © is a constant (nol 


a the power spectrum is still given by 
= (nA*/2)[ fale) + fal—©)] 
ier transforming (0 


(Hint: Time-average the autocorrelation function before Fouri 

obtain Syy(e).] Sond 
(b) Find the total power in X(d) 

of the density function fac). 

7-53 Find the rms bandwidth of the power spectrum : 


Syx(o) = U/L + (o/W)?Y? 


Syx(e) 


and show that it is independent of the form 


where WV > Ois a constant. 
7-54 Work Problem 7-53 for the power spectrum 


Syx(w) = w7/(1 + (a/WwyyP 


7-55 Work Problem 7-53 for the power spectrum 
Syy(w) = UTE + (w/wyy4 
7-56 Work Problem 7-53 for the power spectrum 
Spx) = @2/L1 + (o/WYT? 
67.57 Generalize Problems 7-53 and 7-58 by finding the rns bandwidth of (he 
power spectrum 
Syx(v) = I/L1 + (@/I¥)? 
where N > 2 is an integer. 
*7.58 Generalize Problems 7-54 and 7-56 by finding the rms bandwidth of the 
power spectrum 
Syx(w) = 7/1 + (oy y 
where N 2 3 is an integer. 
7-59 Assume a random process has a power spectrum 
4-—(w7/9) Jw] s6 
Sxx(v) = cau tais 
0 elsewhere 
Find (a) the average power, (b) the rms bandwidth, and (c) the autocorrelation 


function of the process. 
7-60 Show that rms bandwidth of a lowpass random process X(t), as given by 
(7.1-22), can also be obtained from 

—{} d?Ryx(t) 


eae SS WAY z 


Ry) dt 


a0 


where Ry y(t) is the autocorrelation function of X(1). 
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9-61 A random process has the autocorrelation function 
Ryx(t) = B cos? (aq t) exp (— Wr) 


where B. wy, and W are positive constants. 
(a) Find and sketch the power spectrum of X(t) when ap is al least several 


times larger than HW. 
(b) Compute the averige power in the lowpass part of the power spectrum. 


Repeat for the bandpass part In crich case ASSUME Oy) P WwW, 

*7.62 Generalize Problem 7-61 by replacing cos? (qt) with cos’ (avg t) where 
N > Ois an integer. What is the resulling power spectrum when N is (a) odd, and 
(b) even? 

*7.63 The product of a wide-sense stationary gaussian random process X(t) with 
itself delayed by T seconds forms a new process Y(t) = X(NX(t- T). Determine 
{a) Uhe wuitocorrektion finetion, and (pb) the power spectrum of yr). (Mine: Use 
the fact that EDX {X, N,X4) = ELX |X, JELXy XaJ + EUX ,X JEUX. Xa) + 
EEN XJELA Xa) - QELX ,JELX JELN \JELX 4) for gaussian random variables 
NX, N,N, and X4. (Thomas, 1969, p. 64,)} 

7-64 Find the cross-correlation function Ryylt, C+ 1) and cross-power spectrum 
Syy() for the delay-and-multiply device of Problem 7-63. {Hint: Use the fact 
that EQN ,N,X)] = ELX ,JECX2 X35) + ELX JECX Xi] + ELX sJELX 1X2] - 
QELN JELX ETN 3] for three gaussian random variables X,, X2, and X3- 
(Thomas, 1969, p. 64,)} 

7-65 Hf X(N) and Y(f) are real random processes determine which of the following 
functions can be valid. For those that are not, stale at least one reason why. 


(b) [Ryrlt)) SAV RxxOR ys) 


(a) Syy(cu) = 6/06 + 70") 


(a) Ry y(t) = exp (- [th 
(c) Ry y(t) = 2 sin (3t) 


4exp(—-3{el) 


1+? (J) Syylo) = J4 ju? 
) 


(e) Sy yl) = 

(y) Syyten) = 185(c2) 
9-66 Form the product of two statistically independent jointly wide-sensc sta- 
tionary random processes X(t) and Y(A) as 

wi) = XY) 

Find general expressions for the following correlation functions and power 
spectrums in terms of those of X(t) and Y(0: (a) Rywlh 
(b) Ryn. 6+ 1) and Syir(eo), and (c) Rix O+ t) and Sy x(@). (d) If 


Ryxlt) = (WV /a)Sa(, 1) 
and 
Ryylt) = (IY, /n)Sa( Hy 0) 


with constants 1) > W,, find explicit functions for Ryw(t, 6+ t)and Syl). 


t+) and Sip wl), 
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7-67 An cngincer is working with the function 
Ryy(t) = PCL + 1) exp (— W?t?) 


where P > O and W > 0 are constants. He suspects that the function may not be 
a valid cross-correlalion for two jointly stationary processes X(t) and Y(f, as he 
has been told. Determine if his suspicions are (ruc. (Hint: Find the cross-power 
spectrum and sec if it satisfies properties (7.3-16) through (7.3-18).] 

7-68 A wide-sense stationary proccss X(t) is applicd to an ideal differentiator 
having the response Y(/) = dX(o/dt. The cross-correlation of the input-output 
processes is known to be 

Ryy(t) = dRyx(ti/dt 


(a) Determine 8yy(w) and Syx(w) in terms of the power spectrum Syx(tv) of 
X(t). 

(b) Since Syy(co) must be real, nonnegative, and have even symmetry, what 
are the prapertics of 8yy(en)? 
7.69 The cross-correlation of jointly wide-sense stationary processes X(f) and 
Y(t) is assumed to be 

Ryy(t) = Bult) exp (~ Wt) 

where B > O and W > 0 are constants. 

(a) Find Ry,x(t). 

(b) Find Syy(w) and Sy(«). 
7-10 Work Problem 7-69 for the function 


Ryy(t) = Bu(t)t exp (— Wt) 


9-11 Vhe cross-power spectrum for random processes X(t) and Y(t) can be 
wrilfcn as 

Sy) = Syx(C)I() 
where Syx(cw) is the power spectrum of X(t) and H(m) is a function with an 
inverse Fourier transform h(t). Derive expressions for Ryy(t) and Ry,(t) in terms 
of Ry x(t) and A(t). 
7-72 The power spectrum of a bandpass process X(f) is shown in Figure P7-72. 
X() is applied to a product device where the second multiplying input is 
3 cos (v9). Plot the power spectrum of the device's oulpul 3X(1) cos (ta 1). 


S xxl) 
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cay LW ~wy t W iy W ty | 2 Figure P7-72 


on : _~ 


| 
| 


204 PROBABILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 


7-73 Let the “carrier” Ag cos (wot) in Figure 7.5-5 be modified to add a phase 
random variable © so that Y(t) = Ag X(t) cos (wot + ©). If © is uniformly dis- 
tributed on (0, 2) and is independent of X(t), find Ryy(t, t + 1) and Sy,(w) when 
X(t) is wide-sense Stationary, 


7-74 Assume a stationary bandpass process X(t) is adequately approximated by 
the power spectrum 


Sxx(@) = Pua — woo — wo) exp [~(w — Wo)7/b] 
+ Pul(—w — woX—w — wo) exp [—(w + @9)?/b] 


where wo, P>0, and b>0O are constants. The product Y(t) = X(t) cos (wo!) is 
formed. ‘ ig 


(a) Find and sketch the power spectrum of Y(t). 
(b) Determine the average power in X(t) and Y(t), 
n 
eis Compute (he power spectrum of the complex process of Problem 6-55. 
7-76 Let X(t) and Y(J) be statistic 


ally independent processes with power spec- 
trums 


Syx(@) = 25(w) + 1/[1 + (w/10)?} 
and 


Syylo) = 4/01 + (w/2)?} 
A complex process 


Z(t) = (X(t) + JY (Q)] exp (jw 0) 


is formed where wy is a constant much larger than 10. 
(a) Determine the autocorrelation function of Z(t). 
(b) Find and sketch the power spectrum of Z(t). 


CHAPTER 


EIGHT 
LINEAR SYSTEMS WITH RANDOM INPUTS 


8.0 INFRODUCTION 


A large part of our preceding work has been aimed at describing a random signal 
by modeling it as a sample function of a random process. We have found that 
time domain methods based on correlation functions, and frequency domain 
techniques based on power spectrums, constitute powerful ways of defining the 
behavior of random signals. Our work must not stop here, however, because one 
of the most important aspects of random signals is how they interact with linear 
systems, The knowledge of how to describe a random waveform would be of little 
value to a communication or control system engineer, for example, unless he was 
also able to determine how such a waveform will alter the desired output of his 
system, 

In this chapter, we explore methods of describing the response of a lincar 
system when the upplied waveform is random. We begin by discussing some basic 
aspects of linear systems in the following section. Those readers well-versed in 
linear system theory can proceed directly to Section 8.2 without loss. For others, 
the topics of Section 8.1 should serve as a brief review and summary. 


8.1 LINEAR SYSTEM FUNDAMENTALS 

In this section, a brief summary of (he basic aspects of linear systems is given. 
Attention will be limited to a system having only one inpul and one output, or 
response, as illustrated in Figure 81-1. [is assumed (hat the input signal x(t) and 
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Linear 


Input x() system 


Output (4) 


Ati) 
(a) 


Input eC) Output yl) 


my Figure 8.1-1 (a) A general single-input single- 
Mw) output linear system, and (h) a similar linear, 
(b) time-invariant (LTT) system, 


the response y(f) are deterministic signals, cven though some of the topics dis- 
cussed apply to random waveforms, Which topics are applicable to random 
signals will be made clear when they are used in later sections. 


The General Linear System 

Clearly, the linear system (Figure 8.1- ta) will, in general, cause the response y(t) 
lo be different from the input signal x(¢). We think of the system as operating on 
x(t) to cause y(t) and wrile 


y(t) = LEx()] (8.1-1) 


Here L is an operator representing the action of the system on x1). 

A system is said to be linear if its response to a sum of inputs x,(f), 1 = [, 
2,..., N, is equal to the sum of responses taken separately. Thus, if x,(1) causes a 
response y(t), 1 = 1, 2,..., N, then for a lincar system 


N N N 
0 = uf 3 24x. | = Ya, Lx] = Daa syall) (8.1-2) 
awl 


awl nas} 

must hold, where the «, are arbitrary constants and N may be infinite. 

From the definition (2.3-2) and properties of the impulse function we may 
wrile 

xa 
x() = | x(E)d¢t — €) dg (8.1-3) 

By substituting (8.1-3) into (8.1-1) and observing that the operator operates on 
the time function, we obtain 


vo = L(x()] = | x(€)t — &) is] = (" x(OLLS(t — OY) ds (8-1-4) 


We now define a new function f(t, €) as the impulse response of the linear system, 


that is, 


L[S(t — €)] = Mt 6) (8.1-5) 
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Equation (8.1-4) becomes 


vom | xem 8 a (.1-6) 


which shows that the response of a gencral linear system is completely deter- 
mined by its impulse response through (8.1-6). 


Linear Time-Invariant Systems 


A general lincar system is said to be also time-invariant if the form of its impulse 
response h(t, ¢) docs not depend on the time that the impulse is applied. Thus, if 
an impulse 6(), occurring at t= 0, causes the response A(t), then an impulse 
5(t — €), occurring at ¢ = ¢, must cause the response h(t — €) if the system is (ime- 
invariant. This fact means that 


h(t, & = h(t — &) (8. 1-7) 
for a linear, time-invariant system, so (8.1-6) becomes 
Xt) = | x(E)h(t — 2) dg (8.1-8) 
-2o 


Equation (8.1-8) is known as the convolution integral of x(t) and h(¢); it is some- 
times written in the short form 


y(t) = x(t) * A(t) (8.1-9) 


By a suitable change of variables, (8.1-8) can be put in the alternative form 


= { h(E)x(t — &) dé (8.1-10) 
Time-Invariant System Transfer Function 


Either (8.1-8) or (8.1-10) shows that a linear time-invariant system is completely 
characterized by its impulse response, which is a temporal characterization. By 


Fourier transformation of y(t), we may derive an equivalent characterization in 


the frequency domain. Hence, if X(w), Y(w) and H(w) are the respective Fouricr 
transforms of x(#), 32), and A(), then . 


¥(w) = {" ye de = [" is x@hte — 8) deem au 


= [" x} (" ne — enFa-® ate“ dé 


= [" x(@)H(w)e Fe dé = X(w)H(«) (8.1-11) 


-ao 
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With x(t) = exp Geol) as the inpul we must have an output yl) = Manta 
from (81-13). Hence, dy(O/dt = H(w)jex(t) and 


The function H(w) is called the transfer function of the system, Equation 
(8.1-1f) shows that the Fourier transform of the response of any linear time- 
invariant system is equal to the product of the transform of the inpul signal and 
the transform of the network impulse response. 

In the actual calculation of a transfer function for a given network, an alter- 
native definition based on the response of the system to an exponential signal 


x(t) = - Ta jax()) + E(x) 


Finally, we solve for H(«w): 


= elt 81-12 | 

: may be more convenient. It can be shown (Thomas, 1969, p. 142, or Papoulis, < 
1962, p. 83) thatt SS SS ee 
Lfe"] _ a) 
ace a x(0) Bei) Idealized Systems 
where To simplify the analysis of many complex systems, it is often eae) : 

e approximate the system’s transfer function H(w) by an idealized one. Idea ia 

. Hi) = Lfel™] Ghia) transfer functions are illustrated in Figure 8.1-3a for a lowpass system; 4 mh 

pS i et alt RRA A ory case the ideal: 

a An example serves to illustrate the determination of H(w) by means of (8.1-13), : to a highpass system and (¢) applies (o a bandpass system. In every ¢ . bi nd and 

E ie ized system has a transfer function magnitude that is Mat within its passband ¢ 

: ———— ——_______# 

‘e Example 8.1-1 We find H(w) for the network shown in Figure 8.1-2, By al W(w)| of Olu) 
a ussuming a clockwise current i (and no loading in the output circuit), we oN 

e havet i 
x) = LAs yo i 

dt a & 
ney “wn 
But y(t) = iR so By (a) ae) 
é 
di Ln) ” Ww oF Of) 
Ate Reeat _ ts Hort ote 
and aa | 
L dy(t nen ene eh on oe ns on ee se 
x0) = =O 4 yy ove : 
if os R dt sO 
(hb) ate 


ft ft should be carefully observed that (8.1-13) holds only for x(t) given by (8.1-12); that is, for an 
exponential waveform, 


t Lin the network is an inductance and should not be confused with L above, which stands for a 
linear system operator, 


Viwil or 0(w) 


aicaal 


“Wy oo Wy w 


~ 


~~~ 0) 
Input x(r) R Output y(1) Figure 8.1-2 A linear time-invariant network. [Re- 
produced from Peebles (1976) with permission of pub- 
lishers Addison-Wesley, Advanced Book Program] 


Figure 8.1-3 Ideal system transfer functions. (a) Lowpass, (b) highpass, and (c) bandpass i be 
(Reproduced from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book 


Program.) 
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vero outside this band; its midband gain is unity and its phase O(c) is defined lo 
be a linear function of frequency. 

In replacing an actual system with an idealized one, the latter would be 
assigned a midband gain and phase slope that approximate the actual valucs. 
‘The bandwidth W (in lowpass and bandpass cases) is chosen according to some 
convenient basis. For example, 1 could be made equal to the 3-dB bandwidth of 
the actual system, or alternatively, it could be chosen to satisfy a specific require- 
ment. An example of thé latter case is considered in Section 8.5 where W, called 
noise bandwidth, is selected to cause the actual and ideal systems to produce the 
same oulput noise power when each is excited by the same noise source. 


Causal and Stable Systems 


‘To complete our summary of basic topics in linear system theory, we consider 
two final items, 

A lincar time-invariant system is said to be causal if it docs not respond prior 
to the application of an input signal. Mathematically, this implies (9 = 0 for! < 
to if x(t) = 0 fort < fo, where fg is any real constant. From (8.1-10), this condi- 
tion requires (hat 

A(t) = 0 for <9 (8.1-15) 

All passive, linear time-invariant networks that can be constructed will satisfy 
(81-15), As a consequence, w system satisfying (8.1-15) is often called physically 
realizable. 

A linear time-invariant system is said to be stable if its response lo any 
bounded input is bounded; that is, if |x| <M, where M is some constant, then 
Lop < Ml for a stable system where | is another constant independent of the 
input. By considering (8.1-10), it is readily shown that 


l= Ic [h()| di < © (8.1-16) 


—% 


will ensure that a system having the impulse response h(t) will be stable. 


8.2 RANDOM SIGNAL RESPONSE OF LINEAR SYSTEMS 


With the preceding summary of linear system theory in mind, we proceed now lo 
deternine characteristics of the response of a stable, linear, time-invariant system 
as illustrated in Figure 8.1-1b when the applied waveform is an ensemble member 
x(t) of a random: process XU), We assume in all work that the system's impulse 
response A(t) is a real funetion.t In this secon we restrict our allention to tempo- 
ral characteristics such as mean value and mean-squared valuc of the response, 
its autocorrelation function, and applicable cross-correlation functions. Spectral 


characteristics are developed in Section 4.4. 


t All vealeworld networks have real impulse responses, 
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System Response—Convolution 


Even when x(f) is a random signal, the network’s response j(f) is given by the 
convolution integral: 


MO = | x(Bh(t = 8) dg (8.2-1) 
or 
= { h(g)x(t — ¢) ds (8.2-2) 


where /i(t) is the network's impulse response. 

We may view (8.2-2) as an operation on an ensemble member x(t) of the 
random process X(t) that produces an cnsembie member of a new process Y(f). 
With this viewpoint, we may think of (8.2-2) ns defining the process Y(f) in terms 
of the process X(f): 

o 


Y() = | WE)X(t — &) de (8.2-3) 


Thus, we may envision the system as accepling the random process X(t) as its 
input and responding with the new process Y(t) according to (8,2-3). 


Mean and Mcan-Squared Value of System Response 


We may readily apply (8.2-3) to find the mean value of the system's response. By 
assuming X(t) is wide-scnse stationary, we havet 


te ' 
el h(g)X(t — ¢) ic| 


{ N(Z)ELX(t — 6)) de 


ECY(1)] 


=X . h(é)dé= % — (constant) (8.2-4) 


-o 


+t Itis known (Cooper and McGillem, 1971, p. 169) that the operation 


e| { WOOK?) a] = [enon dt 


' 
is valid, where 1/(1) is some bounded function of a random process [on the interval (1,, €,)) and A(q is 
n nonrandom time function, if 


a 
{ ELEWIOL aad dt < oo 


where 1, and ¢, are real constants that may be infinite, This condition is satisfied in all plysical cases 
if 1¥(1) is wide-sense stationury because 1¥(1) will be bounded and the systems are stable (sce (8.1-16)]. 


| 
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a expression indicates that the mean value of Y(t) equals the mean value of 
(t) mes the area under the impulse response if X(t) is wide-sense stationary. 
For the mean-squared value of Y(t), we calculate 


ECY*()] = el [ WS )X(E— 84) de [ W(S.)X(t ~ 22) a,] 


Cs) ao 
= ie [7 exxw — FX (0 — Sa)VWCS aS 2) do, de. (8.25) 
If we assume the input is wide-sense stationary then 


ELX(0 = E,)X(t = GN) = RaalEs — &2) (8.2-6) 
and (8.2-5) becomes independent oft: 


ST fe @ wo . 
Yo = ELY%(N)] =| | Real — SaaS (Sa) de, dee (8.2-7) 


= J- 


Although this expression gives the power in Y(t), it may be tedious to calculate in 
most cases. We develop an example of its solution for a simple case. 


mem merece 


Example 8.2-1 We find Y? for a system having white noise at its input. Here 
Rxx(b, — 62) = (Mo/2)5(E - 3) 


where 4, is a positive real constant. From (8.2-7): 
1g See aa (ae ren 
ie ic [ bt ol DAE, = Eads) dey MEd) de 


= (40/2) {. W(E2) dea 


Outpul power becomes proportional to the area under the square of h in 
(t) 


Autocorrelation Function of Response 
Let X(t) be wide-sense stationary, The autocorrelation function of Y(t) is 


Ryrlt, € +t) = ELV (0 + 9] 
= el [” meixe ~ 8) de, if A(Sa)X(t + t — 22) ue] 


= ie [7 exe — Oy )X(E +t Sa)IME Ea) do, dey (8.2-8) 
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which reduces to 


Ry p(t) = [. J * Ryale +E, — EAMG MED) des Ba be?) 


-w J-w 


- because X(t) is assumed wide-sense stationary. 


Two facts result from (8.2-9). First, Y(0) is wide-sense stationary if X(t) is 
wide-sense stationary because Ry,(t) does not depend on f¢ and ELY(t)) is a con- 
stant from (8.2-4). Second, the form of (8.2-9) shows that Ryy(t) is the two-fold 
convolution of the input autocorrelation function with the network's impulse 
response; that is : 


Ryy(t) = Ryx(t) * 1(—1) * A(t) (8.2-10) 


Cross-Corrclation Functions of Input and Output 


The cross-correlation function of X(t) and Y(t) is 


Revlts t+ 1) = ELX(QYC +O] = g| x0 [" WQ)X(C-+ t= 8) is| 


= | ELX()X(t + t — MACE) de (8.2-11) 
If X(t) is wide-sense stationary, (8.2-11) reduces to 
Ryy(t) = | Ryxlt — OCG) de (8.2-12) 
which is the convolution of Ry x(t) with A(t): 
Ryy(t) = Ryx(t) * AG) (8.2- 13) 
A similar development shows that 
Ryx(t) = | Ryxlt — E)h(— 8) de (8.2-14) 
or 
Ry x(t) = Ryx(t) * h(—7) (8.2-15) 


From (8.2-12) and (8.2-14), it is clear that the cross-correlation functions 
depend on t and not on absolute time t. As a consequence of this fact X(t) and 
Y(t) are jointly wide-sense stationary if X(¢) is wide-sense stationary, because we 
have already shown Y(t) to be wide-sense stationary. 

By substituting (8.2-12) into (8.2-9), autocorrelation function and cross- 
correlation functions are secn lo be related by 


Ry y(t) = | Ryylt + EAE.) doy (8.2-16) 
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or 
Ryy(t) = Ryy(t) * W(—1) (8.2-17) 
A similar substitution of (8.2-14) into (8.2-9) gives 


ao 


Ryy(t) = { Ryy(t — Es)i(Ex) de (8.2-18) 


or 
Ryy(t) = Ryx(t) * h(t) (8.2-19) 


Example 8.2-2 We shall continue Example 82-1 by finding the cross- 
correlation functions Ryy(t) and Ryx(t). From (8.2-12) 


Ryy(t) = iN (A of DHE — ONE) de 


= (MM o/2)h(t) 
From (8.2-14) 
Ryglt) = | (A%o/2)8(t — S)h(— 6) dg 
= (AM o/2)l(—1) = Rev(-1) 


These two results are secn to satisfy (6.3-16), as they should. 


8.3 SYSTEM EVALUATION USING RANDOM NOISE 


ation of the foregoing theory can be immediately developed; it 


A practical applic 
ation function of (8.2-12). Suppose we desire to find 


is based on the cross-correl 


the impulse response of some linear time-invariant system. If we have available a 
broadband (relative to the system) noise source having a flat power spectrum, — 
and a cross-correlation measurement device, such as shown in Figure 6.4-1, h(t) 


can casily be determined. 
For the approximately white noise source 


‘ Rese) > (2)500 (8341 


With this noise applied to the system, the cross-correlation function from (8.2-12) 


or Example 8.2-2 becomes 


Ryle) ® lc (=) — Ee) dé 


3 (<2) (8.3-2) 


Tans 


is 


rar 


2B Doren 
v. 


Att) 


System 
Aq) 


Cross-correlation 
measurement 
system 


A 
Ryylr) 


Figure 8.3-1 A method for finding a system's impulse response. [Reproduced from Peebles (1976) with 
permission of publishers Addison-Wesley, Advanced Book Program.) 


or 
h(t) (F)Ranto (8.3-3) 


Since a measurement Ryy(t) of Rxy(t) can be obtained from the cross-corrclation 
measurement device, (8.3-3) gives us a measurement Ait) of h(x) 


A(t) = (Faas = h(t) (8.3-4) 


Figure 8,3-1 illustrates the concepts described here. 


8.4 SPECTRAL CHARACTERISTICS OF SYSTEM RESPONSE 


Because the Fourier transform of a correlation function (autocorrelation or cross- 
correlation) is a power spectrum for wide-sense stationary processes, it would 


‘seem that if Ry,(t) is known for the input process one can find Ryy(t), Ryy(t), and 


Ry,x(t) as described in Section 8.2 and therefore obtain power spectrums by trans- 
formation. Indeed, this approach is conceptually valid. However, from a practical 
standpoint the integrals involved may be difficult to evaluate. 

In this section an alternative approach is taken where the desired power 
spectrum involving the system's response is related to the power spectrum of the 
input. In every case, the input process X(f) is assumed to be wide-sense station- 
ary, which, as previously proved, means that Y(t) and X(t) are jointly wide-sense 
stationary. 


Power Density Spectrum of Response 


We show now that the power density spectrum Syy(w) of the response of a linear 
time-invariant system having a transfer function H(w) is given by 


Syy(@) = 8xx(w)| H(@) |? (8.4-1) 


where Syy(w) is the power spectrum of the input process X(t). We call | H(w) |? the 

power transfer function of the system. ; 
The proof of (8.4-1) begins by writing Sy;(w) as the Fourier transform of the 

output aulocorrelation function 


Syy(o) = { Ryy(t)e 2" dt (8.4-2) 


¥ 
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On substitution of (8.2-9), (8.4-2) becomes 


Syy(@) = [ nea [| “H(G,) [Pret +o, — E)e' dt dé, dé, (8.4-3) 


The change of variable € = + + €, — €,, dé = dr, produces 


t) 


Syy(w) = [ WE els dé, (iB h(Ga)e “M8 de, i Ryx(S)e“/"% dé (8.4-4) 


These three integrals are recognized as H*(w), H(w), and 8xx(@) respectively, 
Hence 


Syy(co) = H*(w)H(w)8 xx(@) = 8xx(w)| H(w)|? (8.4-5) 


and (8.4-1) is proved. 


The average power, denoted Pyy, in the system's response is readily found by 
using (8.4-5); 


| a 
Pry => { 8xx(w)| H(w) |? dw (8.4-6) 


nner 


Example 8.4-1 The power spectrum and average power of ‘the response of 


a ees of Example 8.1-1 will be found when X(t) is white noise for 
whic 


: NY 
Sxx(@) = rae 


Here H(w) = [1 + GwL/R)]7' so 


I 
2 = oe OO 
| H(cw) |? = CUNT, 
and 
Syy(w) = 8xx(w)| H(w)|? “T+ (@L/RP a ma 


Average power in Y(t), from (8.4-6), is 


1 fe Nef? dw VR 
Pyy=— | Sy(w)dw=—2 | ———_..7 2" 
an rv(a) da = -o 1 +(wL/RY 4L 


after an integral from Appendix C is used. 
Asa check on the calculation of Pyy, we note that (pair 15, Appendix E) 


1 


A(t) = (R/L)u(e~*!* + H(w) = 1 + (j@L/R) 
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for this network, and, using the result of Example 8.2-1, we get 


=F WMo\ f% R * om 2hell _ AOR 
Pyy =} -(4 BNE im 41. 


The two powers are in agreement. 


Cross-Power Density Spectrums of Input and Output 
It is easily shown (see Problem 8-42) that the Fourier transforms of the cross- 
correlation functions of (8.2-12) and (8.2-14) may be written as 
Syy(@) = Syx()H(w) (8.4-7) 
Syx(o) = Syy()H(—w@) (8.4-8) 


respectively, 


8.5 NOISE BANDWIDTH 


Consider a system having a lowpass transfer function H(w). Assume white noise 
is applied at the input. The power density of this white noise is o9/2 where 9 
is a real positive constant. The total average power emerging from the network is 
[from (8.4-6)] 


| w A . 
Pe is (=) E(w) |? deo (8.5-1) 


By assuming the system impulse response is real,t | (cu) |? will be an even func- 
tion of w and (8.5-1) can be written 


Bie | Titata 8.5.2 
ae | FH (cw) |? dw (8,5-2) 
0 


Now consider an idealized system that is equivalent to the actual system in 
the sense that both produce the same output average power when they both are 
excited by the same white noise source, and both have the same value of power 
transfer function at midband; that is, | H(0)|? is the same in both systems. The 
principal difference between the two systems is that the idealized one hits a rect- 
angularly shaped power transfer function | 1/,(c) |? defined by 


| #7(0) |? Jw] < Wy 


22, 
| H,(c)[? = {b ew. (8.5-3) 


¢ The impulse response of any physical system is always real. 


; 
a 
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is a positive constant selected to make output powers 0 the (wo 


where Wy ted t | 
in the idealized system Is 


systems equal. The output power 


° es Mol HO) |? Wy 
1 (? (A% 2 ~~ H(0)}? ig = (8.5-4) 
oe mGe! H,(w)[? da 5a Ks | H(O)V' « aa 


By equating (8.5-2) and (8.5-4), we require that W, be given by 


_ fac Ley dee 
Me = "TARO? 


IW, is called the noise bandwidth of the system. 


ee 
_ 


Example 8.5-1 ‘The noise bandwidth is found for 


transfer function 


a system having the power 


j 


— 
{Hea = Tt (wf ¥)? 


¢ W is the 3-dB bandwidth in radians per second. Here | /1(0) \? = I, so 


mW? la Si (3) Wr 
y= ———; = W tan — = 
Wh i Wit o Wier a 


1 3-dB bandwidth by 


wher 


‘This expression shows that Wy is larger than the systen 


a factor of aboul 1.57. 


oe 


ve development for a bandpass transfer function with a 


it will be found that 


fo | H(w)? dw 


Wy = Too) F 


If we repeat the abo 
eenterband frequency Mo 


(8.5-6) 


der exercise (sce Problem 8-45). The devclop- 


% i it is left as a rea } 
Proof of this result asa A easecanis 


ment also provides 4 simple expression for output noise pow 
bandwidth: 


mia 3 
Pyy = a | H(wo)\? Ww (8.5-7) 


For a lowpass filter, (8.5-7) applics by letting > = 0. 


chet 
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*8. BANDPASS, BAND-LIMITED, AND 
NARROWBAND PROCESSES 


A random process N(t) will be called bandpass if its power density spectrum 
Syn(w) has its significant components clustered in a band of width W (rad/s) that 
docs not include w = 0, Such a power spectrum is illustrated in Figure 8.6-1a.f 
Our definition does not prevent the power spectrum from being nonzero at 
o) = 0; it only requires that Swx(0) be small in relation to more significant values, 
so as to distinguish the bandpass casc from a lowpass power spectrum with sig- 
nificant peaking at higher frequencies. 

All subsequent discussions in this section will relate to special forms of 
bandpass processes. 


*Band-Limited Processes 


If the power spectrum of a bandpass random process is zero outside same fre- 
quency band of width W (rad/s) that docs not include @ = 0, the process is called 
band-limited. The concept of a band-limited process forms a convenient approx- 
imation for physical processes that often allows analytical problem solutions that 
otherwise might not be possible. A band-limited bandpass process power spec: 
(rum is illustrated in Figure 8.6-1). 


+ Power spectrums arising in physical systems will always decrease as frequency becomes sulli- 
ciently large, so a suitable value of W can always be found. For example, bY could be chosen to 
include all frequencies for which Syq(e) 2 O.1Syaltrg) Where tg is some convenient frequency neat 
where Syq{tw) has its largest magnitude (see Figure 8.6-1). 


- 


eae 


=We 0 
a 
(b) 
Figure 8.6-1 Power density spectrums (a) for a bandpass random process and (b) for a band-limited 
bandpass process. 
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*Narrowband Processes 


A band-limited random process is said to be narrowband if W € wo, where wo is 
some conveniently chosen frequency near band-center or near where the power 
Spectrum is at its maximum, A power spectrum of a narrowband process is 
sketched in Figure 8.6-2a. A typical sample function, if viewed on an oscilloscope, 
might took as shown in (b). The appearance of n(t) suggests that the process 
might be represented by a cosine function with angular frequency wy and slowly 
varying amplitude and phase; that is, by 


N(t) = A(t) cos [wy t + O(t)] (8.6-1) 


where A(t) is a random process representing the slowly varying amplitude and 
©(t) is a process representing the slowly varying phase, Indeed this is the case, 
and, for the important practical case where N(t) is gaussian noise, it is known 
that A() and Q(t) have Rayleigh and uniform (over 2m) first-order probability 
density functions respectively. The processes A(t) and (0) are not statistically 
independent when N(t) is gaussian (Davenport, 1970, p. 522, or Davenport and 
Root, 1958, pp. 161-165), but for ‘ny one instant in lime the process random vari« 
ables are independent, 


In some problems, (8,6-1) is a preferred representation for N(t). For others, it 
is convenient to use the equivalent form 


N(t) = X(t) cos (wot) — Y(t) sin (wot) (8.6.2) 
S www) 
W <u, ev 
“We 0 Wo w 
(a) 
Carrier with (ce) Randomly 


randonily fluctuating 
fluctuating envelope 


Figure 8.6-2 (a) A power spectrum 
of a narrowband random process 
N(t) and (b) a typical ensemble 
member a(t), (Reproduced from 
Peebles (1976) with permission of 
publishers Addison-Wesley, Ad- 
(h) vanced Book Program 
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where the processes X(t) and Y(t) are given by 
X(t) = A(t) cos [O(t)] (8.6-3) 
Y(t) = A(t) sin [O(t)] (8.0-4) 
Expressions relating A(t) and O(1) lo X(‘) and Y(s) are 
A(t) = JX) + 70) (8.6-5) 
@(t) = tan! LY()/X(0) “. (8.6-6) 


*Propertics of Band-Limited Processes 


The representations (8.6-1) and (8.6-2) are actually more general than implied 
above; they can also be applied to any band-limited random process. For the 
remainder of this section we concern ourselves only with (8.6-2). 

Let N(t) be any band-limited wide-sense stationary real random process with 
a mean value of zero and a power density spectrum (hat satisfies 


Syl) # 0 
Sya(w) = 0 


0 < my — W, < lw] <a — 4 + W 
elsewhere (8.6-7) 


where IY, and W are real positive constants. Then N(¢) can be represented by the 
right side of (8.6-2),t where the random processes X(t) and Y(t) have the follow- 
ing properties: 


(1) X(t) and Y(0) are jointly wide-sense stationary (8.6-8) 

(2) E[X(t)] =0 ELY(s)] =0 (8.6-9) 

(3) EL X?(t)] = ELY7(t)] = ELN7(s)) (8.6-10) 

(4) Ryxx({t) = : [sto cos [(w — Wo)t] dw (8.6-1 1) 
0 

(5) Ryy(t) = Rxx(t) (8.6-12) 

(6) Kyy(t) = - eo sin ((w — wo)t) dw (8.6-13) 
0 


(7) Ryx(t) = —Ryy(t) 
(8) Ryy(O) = ELX(t) ¥(t)] = 0 


Ryylt) = —Ryy(—1) (8.6-14) 
Ryx(0) = 0 (8.6-15) 


t I we denote the right side of (8.6-2) by N(c) the equality in (%.6-2) must be interpreted in the 
sense of zero mean-squared eeror; thal is A(t) equals A() in the sense that 


EC(N() — N()}?] = 0 
(Ziemer and Tranter, 1976, p. 241). 
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(9) Sxx() = L[Syl@ — (Wo) + Synlo + Wo)] (8.6-16) 
(10) Syl) = Sxxl) (8.617) 
(11) Syy(e) = jL, [Syl — 9) — Swalco + W)J (8.6-18) 
(12) Sys(o) = —Sxr(~) (8.6-19) 


In the preceeding 12 results, Wo is any convenient frequency within the band of 
Syn) Ryxlth Ryplt) Ryylt)s and Ry x(t) are autocorrelation and cross- 
correlation functions of X(/) and Y(0) while Sy (m), Syl), Sxyl)s and Syx() are 
the corresponding power spectrums, and L,[ +] denotes taking the lowpass parl 
of the quantity within the brackets. 

We outline the proofs of the above properties in the next subsection, Here we 
discuss their meaning and develop an example. We sce that in addition to being 
zero-mean (property 2) wide-sense stationary (property 1) processes, X(!) and Y(t) 
also have equal powers (property 3), the same autocorrelation function (property 
5), and therefore the same power spectrum (property 10). Random variables 
defined for the processes X(f) and Y(t) at any onc time are orthogonal (property 
8). If N(t) has a power spectrum with components having even symmetry about 
w = +9, then X(t) and Y(t) will be orthogonal processes (property 6). A conse- 
quence of this last point is that the cross-power spectrums of X(t) and ¥(¢) are 


zero (propertics It and 12). 


Example 8.6-1 Consider the bandpass proccss having the power density 
spectrum shown in Figure 8.6-3a. We shall find Syx(@), Sxy(@), and R,y,(t). 
By shifting Syx(@) by +@Wo and —Wo as shown in (b), we may construct 
Syy(w) according to (8.6-16) as the lowpass portion of Swx(@ — Mo) + 
Syn + Wo) a8 iMustrated in (c). This function also equals Syy(w) by (8.6-17). 
Similarly, we form the difference of the spectrums in (b) to obtain 8xy(w) 
according to (8.6-18) as shown in (d). This function also gives Syx(w) from 


(8.6-19) as shown. 
To find Ryy(t) we apply (8.6-13): 


! vo t Ws . P 
Ryy(t) = a | P sin ((o — @p)t] da = oi | 


wo Wy 


Woe 
sin (x) dx 
-Wit 


P 
= — [cos (I/,t) — cos (W440) 

nt 

P : (Ww, + Wt UR = W,)t 
= —-4COS a a 


cv + We , Wa = We 
— cos a 2 er ae 


op . [(W+ Wot) | [“ - mi) 
=—sin | sin. 
nt 2 2 


ate 
Pa 


W, +W, 


-2wWe -Wo| 0 | We 
-we-W, -Wo t Hy We-W, wot 


(a) 


Saniw - Wo) 
P 


LINEAR SYSTEMS WITH RANDOM INPUTS 223 


aw 


W, 


We" “Wy -W, OW, We two w 


Snwlw + Wo) 


-W, -W, W, W, wW 
(¢) 


-/Syy(w) or /Syx(w) 


(d) 


Figure 8.6-4 Cross-correlation function of Example 8.6-1. 
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Now since IV, + W, = W, we may write this result as 


WP sin (Wr/2) . 
Ryylt) = a gj - 
xrlt) ‘ 4 (ep) sin [(W — 2W,)t/2] 
which is an odd function of ¢ as (8.6-14) indicates it should be. Figure 8.6-4 
illustrates a plot of Ry,(t) for the special case W, = W/6. 

It should be noted that if W, = W/2, corresponding to Sya(v) having 
even components about w = peo, We gel Ryy(t) = 0 for all tr. In this case, 
X(t) and Y(t) are orthogonal processes; they are also independent if N(s) is 
gaussian, yt 
eee 

*Proof of Properties of Band-Limited Processes 


It is a quite long and involved task to prove all 12 properties of band-limited 
processes in detail. Therefore, we shall outline most of the proofs and give the 
details on only a few. 

Property 2 is proved by taking the expected value on both sides of (8.6-2). 
Since N(t) is assumed wide-sense stationary with a mean value of zero, then 
E[X(t)] = 0 and ELY(t)] = 0 are necessary and property 2 follows. 

The sequence of developments leading to the proofs of properties 9 and 4 will 
now be given. We begin by assuming the usual case W, = W/2 (see Figure 8.6-1h) 
and observing that the network of Figure 8.6-5a gives X(t) at its output if the 


ideal lowpass filler has a bandwidth W/2 and if wy > W/2.t We shall assume 
these conditions (rue. Thus 


V(t) = 2N(t) cos (wo t) 
= 2[X(t) cos? (wp t) — Y(t) sin (wo t) cos (wo 1)] 
= X(t) + [X(1) cos (2w9t) — Y(t) sin (2w9 1)] (8.6-20) 


The filler will remove the bandpass process contained within the brackets so that 
only X(t) appears in the output. Next, we develop an expression for Ryx(t,f + 1): 


Ryxlt, (+t) = ELX(Q)X(t + 1] 
= A [ hQwV,(t — u) du ii h(v)V,(t + t ~ v) w| 


w -@ 


= i [ A(Uh(Y)Ryn(t + uv — v)4 cos [wo(t — u)] 


cos [w(t + t — v)) du dv (8.6-21) 


t These ure idealized values based on un ideal product device. Practical values of bandwidth and 
tg may be considerubly different. The ussumption W, = I/2 is for simple definition of filter band- 
width and is nota constraint in properties 9 or 4, 
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Set) deat 
mone Product Fai] lowpass Nay 
_— — : flier 


ae 
2d cos (wy!) Bandwidth = 3 


(a) 


yur Figure 8.6-5 Block diagrams of networks that 
realize (a) XC) and (b) Yq) from a random 
process N(t) = X(0) cos (wal) = Y(t) sin (avy 0). 


Bandwatih -M [Reproduced from leebles (1976) with per: 
mo sin (aay) mission of publishers Addison-Wesley, 
(2) Advanced Book Program.) 


In developing (8.6-21), we have written X(¢) and X(¢-+ 1) in terms of the convolu- 
tion integral involving (1), the impulse response of the lowpass filler, substituted 
V(t) from (8.6-20), and used the fact that N(1) is assumed wide-sense stationary. 
The further reduction of (8.6-21) is lengthy (Peebles, 1976, p. 157) and will only be 
outlined. If the cosine factors are replaced by their exponential forms and if 
Ryplt + — v) is replaced by its equivalent, the inverse (ransform of the power 
spectrum Sy,(w), (8.6-21) becomes the sum of four integrals. It can be shown 
that two of these integrals, the only two involving ¢, are zero. Thus, Ryy(t, ¢ + 1) 
becomes a function of t only and X(t) is therefore wide-sense stationary, proving 
part of property !. The two remaining integrals are used to prove properties 9 
and 4, 

A procedure exactly the same as discussed in the last paragraph can be used 
(o prove first that Y(t) is wide-sense stationary, thereby providing the proof of 
another part of property 1. The development also proves properties 10 and 5; it 
is based on the fact that Y(t) is produced by the operations shown in Figure 
8.6-5b. 

Property 3 next results from use of property 5 with + = 0 and the integration 
of Syx(«) using property 9. 

Properties 11, 6, 8, and the balance of property | are proved by considering 
the cross-correlation function 


Ryylt, £ + t) = ELX() V(t +t] 
4 (" h(u)V,(e — u) du a h(v) V(t + t — v) iv| 


us beat) - 


I 


_ im " ha h(o)Ryy(t + 4 —v)4 cos Lwolt — uy] 


sin [@o(t + t — v)] dv du (8.6-22) 


which is developed in a manner analogous to (8.6-21), Reduction of (8.6-22) as 
discussed carlicr shows (hat Ryy(t, ¢ + t) depends only on t, so that X(q and Y(t) 
are jointly wide-sense stationary (proving property 1); it also proves properties !1 
and 6. Property 8 resulls from property 6 with t = 0. 
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ree 
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Proofs of the remaining propertics, 7 and 12, follow from consideration of the 
autocorrelation function of N(t). It is readily found by using (8.6-2) that 


Ryall. bo) = ELN(ONG + 1) 
= [Rys(t) + Ryylt)]'h cos (1) 
+ (Ryx(t) — Ryv()] Y, cos (29 f + ot) 
= [Ryslt) — Ryx(t)] VY, sin (9 t) 
= [Ryylt) + Kral); sin 29 + Mo 1) (8.6-23) 


Since N(t) is wide-sense stationary by original assumption, its autocorrelation 
function cannot be a function of t. Thus, we require 


Ryx(t) = Ryylt) (8.6-24) 


and 


Ryy(t) = — Ryx(t) (8.6-25) 


in (86-23); these results prove property 12 and the first. part of propertly 7 
Finally, recognizing that Ryy(t) = Ry) for a cross-corrclation function, we 
obtain the second part of property 7, which says that Ryy(t) is an odd function 


of t. 


8.7 MODELING OF NOISE SOURCES 


All our work in this chapter so far has related to finding the response of a linear 
system when a random waveform (desired signal or undesired noise) was applicd 
at its input. In every case, the system was assumed to not contain any internal 
sources. In particular, the system was assumed to be free of any internally gener- 
ated noise. In the real world, such an assumption is never justified because all net- 
works (systems) generale one or more types of noise internally. For example, all 
conductors or semiconductors in a circuit are known to generate thermal noise 
{sce Section 7.5) because of thermal agitation of free electrons.t The question 
naturally arises: How can we handle practical networks that produce internally 
generated noise? The remainder of this chapter is concerned with answering this 
question. ” 

We shall find that, by suitable modeling techniques for both the network and 
for the external source that drives the network, all the internally generated 
network noise can be thought of as having been caused by the external source. In 


of internally generated noise such as shot noise, partition noise, 


t There are many other lypes 
induced grid noise, flicker naise, secondary emission noise, ete. The reader is referred to the literature for 


more detitil (Munford and Scheibe, 1968, van dee Ziel, 1970). 
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effect, we shall replace the noisy practical network with a noise-free identical 
network that is driven by a “ more noisy" source. 
Our work begins by developing models for noise sources. 


Resistive (Thermal) Noise Source 


Suppose we have an ideal (noise-free, infinite input impedance) voltmeter that 
responds to voltages that fall in a small ideal (rectangular) frequency band dw/2n 
centered at angular frequency «. If such a voltmeter is used to measure the 
vollage across a resistor of resistance R (ohms), it is found, both in practice and 
theoretically, that a noise voltage @,(t) would exist having a mean-squared value 
given by 

a y a da (8.7-1) 


Here k = 1.38(10727) joule per Kelvin is Boltzmann's constant,t and T is tem- 
perature in Kelvin. This result is independent of the value of w up to extremely 
high frequencies. (See Section 7.5 where .A79/2 equals 2kTR here. The reader 
should justify this fact as an exercise.) 

Now because the voltmeter does not load the resistor, x(t) is the mean- 
squared open-circuit voltage of the resistoriwhich can be ireated as a vollage 
source with internal impedance R. In other words, the noisy resistor can be 
modeled as a Thevenint voltage source as shown in Figure 8.7-la. An equivalent 
current source is shown in (b) where 


2kT dw 


AG = aayR? = = 


(8.7-2) 


is the short-circuit mean-squared current. 
From Figure 8.7-1a it is found that the incremental noise power dN), delivered 
to the load in the incremental band dw by the noisy resistor as a source is 
_ eR QkTRR, dw 
HT RER) (K+ Ri)? 


(8.7-3) 


The maximum delivered power occurs when RK, = R. We call this maximum 
power the incremental available power of the source and denote it by dN,,; it is 
given by 
kT dew 

AN, = eRO/AR = (8,7-4) 
We see from (8.7-4) that the incremental power available from a resistor source is 
independent of the resistance of the source and depends only on its physical tem- 
perature T. These facts may be used as a basis for modeling arbitrary sources. 


+ Ludwig Boltzmann (1844-1906) was an Austrian physicist. 
{ Named for the French physicist Léon Thevenin (1857-1926). 
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Noisy Noise 
resistor free 


Temperature T, 


Temperature 7; 


Figure 8.7-2 Equivalent circuits for 
two resistors at different temperatures 
in series. 


| 
- | . . . . . . . 
| | Figure 8.7-2 illustrates Thevenin equivalent circuits for the combination. 
' Since the individual resistors as sources may be considered independent, their 
| Figure 8.7-1 Equivalent circuit models of 1 noisy mean-squared voltages add. Hence, 
| = resistor: {a) vollage model and (b) current model, “Tn 4 SH TI 
t Pah a Sans ek J [Adapted from Peebles (1976) with permission of ex(f) bt e3() = en(t) 


publishers Addison-Wesley, Advanced Book Pro- 


gram] By applying (8.7-1) to both sides of the preceding expression, we obtain 


dw 


| ; 
2KLTR, + 1, Ry] = = 2kLTR, + RY] 


fy 


; 
fe 


Arbitrary Noise Sources, Effective Noise Temperature 


Suppose an actual noise source has an incremental available noise power dN,,, 
Open-circuit output mean-squared voltage e(t), and impedance as measured 


between its output terminals of Z(@) = R,(w) + j i i 
: . ° + X ,(w). TI a 
power is casily found to be eaten Sane 


1 = GRC) (8.7-5) 


If we now ascribe all the source's noise to the resistive part R,(w) of its output 


oe by defining an effective noise temperature T, such that (8.7-1) applies 
hen 


= dw 

en(t) = 2kT; Ry(cw) = (8.7-6) 
As with a purcly resistive source, available power is still independent of the 
source impedance but depends on the source's temperature 


in dw 
(Nas =k, on (8.7-7) 


We consider (wo examples that illustrate elfective noise temperature. 


Example 8.7-1 Two different resistors at different physical temperatures are 
placed in series, The effective noise temperature of the series combination as 
a noise source is to be found. 


or 
raw DRit+ TR 
OR, +R, 


Example 8.7-1 clearly shows that effective noise temperature of a source is 
not necessarily equal to its physica! temperature. In the special case where 7) = 
T, = T, then T,= T. More generally, it is true that any passive, two-terminal 
source that contuins only resistors, capacitors, and inductors, all at the same 
physical temperature 7, will have an ellective noise (emperature Tj = 7. (Ziemer 
and Tranter, 1976, p. 471). The next example can be used to illustrate this last 


point. 


Example 8.7-2 We reconsider Example 8.7-1, except we now allow a cupac- 
itor to be placed across one resistor as shown in Figure 8.7-3. 

By superposition, ¢2(1) is the sum of contributions from each resistor as a 
noise source. The mean-squared voltage, denoted e(0, due to the first 
resistor is readily secn to be 

1 | oe ee LC OR 
- mT b JOR Cy | 1 bw? R2CH 


! 
: 
a 


Segeie 
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— 


Temperature 1, 


Vemperature 7; 


_ 


Figure 8.7-3 Equivalent cireuils for 


capacitor. 


a linear, passive. two-lerminal network of Iwo resistors and one 


That duc to the sccond resistor is 


oh) = eX) 


Thus, by applying (8.7-1) lo the two individual resistor mean-squared voll- 


ages, we have 
da 


a. T,R, do 
e2(t) = eat) + eral!) = al RIGA + T; ny = 


Next, we find the output impedance of the network as an overall source by 
imagining the noise sources set to 0. We gel 
R,(twCy) Ry 
4 eee ROOT pS 
Zh) = Rat RV joc) JORG 


1+ w? RIC} 


Ry + 

whiel) has a resistive part 

R,(w) = Ra + eee een 
ho) = 2 tw? RCT 


By applying (8.7-6) to the cquivalent source, WC have 


R dw 
ay te | 
eal!) = 241 Rs a + eal t 


Finally, we equate ey for the actual and equivalent networks lo find Ty: 


2= Re RAL + @? RIC) 


tite ee 
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The preceding example shows that effective noise temperature may be a 
function of frequency. In this case, the available noise power is also frequency 


: dependent. 


Again we sce that T, = T in the above example if T, = 7; = T, as it must 
because it is a linear, passive, two-terminal network with only resistors and a 


capacitor, as noted previously. 


An Antenna as a Noise Source 


In practice, all antennas produce noise at their output because of reception of 
electromagnetic radiation from noise sources external to the antenna.t The 
amount of available noise power dN,, in an incremental band dw depends in a 
rather complicated manner on all the space surrounding the antenna, However, it 
is possible to model the antenna in a simple way by assigning to it an antenna 
temperature T, chosen so that dN,, and T, are related by (8.7-4), Thus, 
(1) 

dN,, = kT, = (8.7-8) 
In general, antenna temperature may vary with frequency. However, in many 
applications T, can be considered constant (with respect to w) because ils varia- 
tion with frequency over a frequency band comparable to that of the desired 
signal being received is often small, : 


Example 8.7-3 A very sensitive meter that is capable of measuring noisc 
power in a (small) frequency band | kHz wide al any frequency «/2n is 
attached to a microwave antenna used in a radio relay link. It registers 2.0 
(107 '8) W when the meter’s input impedance is matched to the antenna so 
that its reading is maximum. We find the antenna (emperature 7,. 

Since maximum power is extracted from the antenna, the power is its 


available power and (8.7-8) gives 
-{8 - 2 
2ndN,, 2n(2)10 2 200 ~ 144.9 K 


EN Emre khan 
T kdw 1,38(10"?*)2n(10*) 1.38 
i 


8.8 INCREMENTAL MODELING OF NOISY NETWORKS 


In this section we shall show how a noisy network can be modcled as a noise-free 
network excited by a suitably chosen external noise source. We also develop 
some measures of the “noisiness” of a network. All our work is applicable to an 


incremental band de. 


t There are many sources of external noise; several of these are described by Peebles (1976, pp. 
463-464). 
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Linear 


two-port Z Z Figure 8.8-1 A linear two-port 
network : 

network driven by uw source of 
impedance Z,. 


Available Power Gain 


Consider first a linear, noise-free, two-port (4-terminal) network having an inpul 
impedance Z,; when the output port is open-circuited. Its oulput impedance, 
found by looking back into its output port, is Z, when being driven by a source 
with source impedance Z,. The source open-circuit voltage is e,(t) and the 
network’s open-circuit output voltage is e,(t). The applicable network is illus- 
trated in Figure 8.8-1, 

The available power, denoted dN,,, of the source is 


ea 
dN,, = (8.81) 


where R, is the real part of Z,. This power is independent of Z,. The available 

power, denoted dN,,,, in the output due to the source is 
4R, 

where R, is the real part of Z,. This power does depend on Z, through its influ- 

ence on the generation of e,(1) but does not depend on the load impedance Z,. 


We define the available power gain denoted G, of the two-port network as the 
ratio of the available powers 


AN gos (8.8-2) 


_ (Nae _ Re 

“dN, Ries) 
When a cascade of M noise-free networks is involved where M = | ey Parana | 
is casy lo see that the overall available power gain G, is the product of available 
power gains G,,, ors 1, 2,..., M, if G,, is the gain of stage m when all preceding 
Stages are connected and treated as ils source (see Problem 8-65). Thus, 


(8.8-3) 


G.= 1G, (8.8-4) 


Equivalent Networks, Effective Input Noise Temperature 


Consider next the case of a linear two-port network with internally generated 
noise, The network is assumed to be driven from a source with effective noise 
temperature T, as shown in Figure 8.8-2a. If G, is the network’s available power 
gain, the available output noise power due to the source alone is 

dw 


dN ios = GaN, = GkT, 


a (8.8-5) 


from (8.8-3) and (8.7-7). 


A 
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Souree temperature noe| MB Fe UN, + ON FUN 
(a) 
Soures temperature 707, —| ae ae on UM yay | AN aa Noe 


Uh} 


ap dw 
KT, ed Noise-free 


network 


UN yo, + AN yo ® U3 


{c) 
Figure 8.8-2 A network with internally generated noise driven from a noise source (a), and equivalent 
noise-free networks (b) and (c). [Reproduced frum Peebles (1976) with permission of publishers: Addison- 
Wesley, Advanced Book Program.] 


Total available oulput noise power dN,, is larger than dN,,, because of inter 
nally generated noise. Let AN,, represent the excess available noise power at the 
output, We shall imagine that AN,, is generated by the source by defining e/fec- 
tive input noise temperature T, as the temperature increase that the source would 
require to account for all output available noise power, It therefore follows that 


AN, = G,kT, (8.8-6) 
2n 


With this definition, the noisy network is replaced by a noise-free network driven 
by a source of temperature T; + Tas shown in Figure 8.8-2, 

It is somewhat helpful to model the available source noise power by use of 
wo inputs, as shown in Migure 8.8-2c, The second input represents the internally 
generated noise due to the network, The representation is convenient in visual- 
izing noise effects when networks are cascaded as illustrated in Figure 8.8-3, By 
equating expressions for output available noise powers in the cascade and equiv- 
alent network, the effective input noise temperature T, of the cascade is deter- 


mined to be 


Ter Tes Tem 
Tet, +e ch top oe (8.8-7) 
‘ G, G,G, G,G,°°* Gye 


where 7;,, and G,,,m = 1, 2,..., M, are the effective input noise temperature and 
available power gain, respectively, for the mth stage when all m— 1 previous 
stages are connected and form its source, 

An especially useful application of (8.8-7) is to the cascade of stages in an 
amplifier. We develop an example. 
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Gu 


Gy Gy Cn 


Network ae Network es 
\ mt 


UN any 


Network 
M 


a Source 
+ ONy 


“is . : . de od 
| AV a AT “ kT an Se kT ot ar 
‘ (ay 
P GGG, Gur 


dNya + Nu 


P Source 


[ (b) 
Figure B.R-3 (a) M. networks in cascade and (hb) the equivalent network. (R 
(1976) with permission of publishers Addison-Wesley, Advanced Book Program.) 


eproduced from Peebles 


Example 8.8-1 ‘The stages in a three-stage amplifier have elfective input noise 


et 
mt temperatures Thy = 1350 K, Ty = 1700 K and 1. = 2600 K. The respective 
bs. available power gains are G, = 16, G, = 10, and Gy = 6. We find the clTec- 
be : tive input noise temperature of the overall amplifier by usc of (8.8-7): 
yC 
te 1700 2600 

T= —+——~ = 13 06.25 + 16.25 

T, = 1350 + re * 1e10) 1350 + 1 + 


N = 1472.5 K 


We sec that, even though T., and T,; are larger than T,,, the contributions to 


ia: T, by the second and third stages are much smaller than that of the first stage’ 
| E because of the gain of previous slages. In general, it is clear from (8.8-7) that 
mat an amplifier should have its lowest noise, highest gain stage first, followed by 


a its next best stage, ctc., for best noise performance. 


A Spot Noise Figures 


‘ [Effective input noise temperature 'r, of a network is a measure of its noise per- 
formance. Better performance corresponds 0 lower values of T,. Another 
mance is incremental or spot noise figure denoted by F and 


| measure of perfor d 
tal available output noise power dN,, divided by the 


defined as the total incremen 


a incremental available output noise power due to the source alone: 

i 

' oy dN _ dN oe + ANoo @ \ Fear (8.8-8) 
$ . AN aos AN os AN jos 

i An alternative form derives from the substitution of (8.8-5) and (8.8-6): 

on Felec (8.8-9 
ld T. ) 


ara. 
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In an ideal network, T, = 0 so F = 1. For any real network, F is larger than 
unity. 

In practice, a given network might be driven by a varicty of sources, For 
example, an amplifier might be driven by an antenna, mixer, attenuator, other 
amplifier,etc. Its spot noise figure istherefore afunction of the effective noise tem- 
perature of the source. However, by defining a standard source as having a stan- 
dard noise temperature To = 290 K and standard spot noise figure Fo, given by 


T, 
Fo=t saci (8.8-10) 
0 
a network can be specified independent of its application. 
When a network is used with the source for which it is intended to operate F 
ae - called the operating spot noise figure and given the symbol F,,. From 


Fmlta (8.8-11) 


J 


Operating and standard spot noise figures can also be developed for a 
cascade of networks (sec Problems 8-66 aird 8-68). 


en 


Example 8.8-2 An engineer purchases an amplifier that has a narrow band- 
width of | kHz and standard spot noise figure of 3.8 at ils frequency of oper- 
ation. The amplificr’s available oulpul noise power is 0.1 mW when ils inpul 
is connected to a radio receiving antenna having an antenna temperature of 
80 K. We find the amplifier’s input effective noise temperature T,, its oper- 
ating spot noise figure F,,, and its available power gain G,. : 

T, derives from (8.8-10): 


T, = To(Fo — 1) = 290(3.8-1) = 812 K 
We can now use (8.8-11) to obtain F,,: 
812 
F=l+ 7 ell. 
i + 80 11.15 
From (8.8-5) and (8.8-6) we add to get total available output noise power: 


dN 7 AN yos + AN xn ad Se 
nu 


so 


2n dNyo *  2n(0.1)1074 


Ge Ne Os 
KT, + T,) deo 1.38(10~ 73812 + 80)2n(10°) © 8.12(10") 


Three 
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8.9 MODELING OF PRACTICAL NOISY NETWORKS 


In a realistic network, the frequency band of interest is not incrementil, There- 
fore such quantities as available power gain, noise temperature, and noise figure 
ure not necessarily constant but become frequency dependent, in general. In this 
section we extend the earlier concepts based on an incremental frequency band to 


include practical networks, by defining average noise temperatures and average 
noise figures. 


Average Noise Figures 


We define average operating noise figure F,, as the total.output available noise ~ 


power N,, from a network divided by the total output available noise power N 
due to the source alone. Thus, ' 


aos 


al Ny 
Fo = Re (8.9-1) 
N gos is found by integration of (8.8-5): 
k wo 
Nw == ; 
ia al T,G, dw (8.9-2) 
We may similarly use (8.8-8) with (8.8-5) to determine Nye! 
N= { dN,, = | Fup INyy = Le Fy, TG, dw (8.9-3) 
0 _ 0 2n Jo *? 
Thus, from (8,9-1) 
pL (8.9-4) 


ve Se TG, dw 


In many cases the source’s temperature is approximately constant. Operating 
average noise figure then becomes 
a fo op G, dw 
PSP G, dw 
An antenna is an example of a source having an approximately constant noise 
temperature (so long as the surroundings viewed by the antenna are fixed). 


Another example is a standard source for which T, = Ty = 290 K is constant. We 
define average standard noise figure Fy as that for which the source is gtandard. In 


this case 
F, = Se FoG, dw ; (8.9-6) 


o” SR G, dw 


as can be shown by repeating the steps leading to (8.9-4). 


T, constant (8.9-5) - 
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Average Noise ‘Temperatures 


From the delinition of effective inpul-noise temperature ‘Ty, it follows that the 
incremental available output noise power from a network with available power 
gain G, that is driven by a source of temperature T, is 


dw 
= . See (8.9-7) 
UN = Gak(T, + 7) 
Total available power is therefore 3 
No = (raw. = aS | GAT, + T,) dw (8.9-8) 
uo o Qn ey 


Next, we define average effective source temperature T, and average effective 
input noise temperature T, as constant temperatures that produce (he same total 
available power as given by (8.9-8), Hence 


(7, + T.) | G, dw (8.9-9) 
0 


By equating (8.9-9) and (8.9-8) on a term-by-term bitsis, we gel 


7 = fo’ T,G, dw (8.9- 10) 


~ §8 G, dw 
and 
pu ee (8.9-11) 
0 G, dw 


If (8.8-10) and (8.8-11) are substituted into (8.9-6) and (8.9-4), respectively, we 
obtain the interrelationships 


T 
Bae fees 8.9-12 
Fost+ Tr ( ) 
Fy, =! + (8.9-13) 


By equating % from these last two expressions, we obtain alternative interrela- 
tionships 


Po =l+ a (Pup — 1) (8.914) 
Tp 
Puy Vb 2 (Fy - 0) (8.9-15) 
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Average effective noise temperature is a very useful concept for modeling 
network noise in a simple way. To demonstrate this fact, note that (8.9-9) can be 
written as 


Nok (ht TG Loo) * Gia dee (8.9-16) 
2n G,(@o) 


where Wo is the centerband angular frequency of the function G,(w). Since G,(w) 
is the available power: gain (or power transfer function) of the nctwork, we 


identify 
Wy = [e Golo) deo (8.9-17) 
G,{o) 
as the noise bandwidth of the network, by analogy with (8.5-6). Rquation (8.9-16) 
becomes 


2 W, 
No = Glwok(T, + Te = (8.9-18) 


which says that actual available output noise power is that due to a source with 
constant temperature T+ T, driving an equivalent noise-free network with an 
ideal rectangular transfer function of bandwidth W,(rad/s) and midband available 
power gain G,(u9). This result represents a very simple network model. 


Modeling of Attenuators 


Consider a source of average effective temperature T, driving an impedance- 
matched lossy attenuator with power loss L (a number not less than onc) at all 
frequencies. The attenuator has a physical temperature T,. It can be shown 
(Pecbles, 1976, P- 463; Mumford and Scheibe, 1968, p. 23) that the average effec- 
live input noise temperature of the attenuator is 


T,=T(L~- 1) (8.9-19) 
From (8.9-12) and (8.9-13) the applicable average noise figures are 


uF 
Poel +a(L-% (8.9-20) 
To 


Paat+g(l=0) (8.9-21) 


Note that if 7), = To oF if T;, = T,, the average noise figure of the attenuator is 
just equal to its loss. 


Model of Example System 


One of the most important applications of the theory of this and the preceding 
two sections is in modeling receiving systems. As illustrated in Figure 8.9-1a, con- 
sider a receiving antenna that drives a receiver amplifier through various broad- 
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Antenna Physical Available power 
temperature T, temperature gain = Galto) 
Te 


Naise-free 
receiver 


Noise-free 
loss L 


kT (L = 1) AT, Wy 
yr an 
(a) 
Gain = Galo) 
KT, Way A Nolse-free 


Tn 


rystem 


(b) 


Figure 8.9-1 A madel of a receiving system (a) and its equivalent (}). [Reproduced fram Peebles (1976), 


with permission of publishers Addison-Wesley, Advanced Book Program.) 


band components having an overall loss L. These components (which may 
include microwave transmission lines, isolators, of other devices) are all assumed 
to have physical temperature T,. The antenna temperature is T, and the receiver 


average effective input noise temperature is Tp. The receiver's noise 


bandwidth is 


Wy and it has a centerband available power gain Galo) We demonstrate that 


the system is equivalent to that shown in Figure 8.9-1b. 


The equivalent system has the same noise bandwidth as the actual system 
and has a centerband available power gain Galwo)/L. It is driven by a simple 
source with system noise temperature Ty. The available oulput noise power in 
the actual system is the sum of the antenna's contribution plus those due to 
excess noises in the attenuator and receiver. By using earlicr models, this noise 


power is 
Ng = ACT + Til 1+ Ty ly SAE 
For the equivalent system 
nant 


By equating the above two expressions, We obtain 


Fo = T + Tle It Teh 


aya 


(8.9-22) 


(8.9-23) 


(8.9-24) 


From (8.9-24), the average effective input noise temperature of the system taken 


at point A in Figure 8.9-la is 
f=T(L—-1)+ Tab 


(8.9-25} 


bent cea Te 


Fo abe cere 


cpa ine ware PtH ae 


ca * 


ff 
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. 9. *IVvEerape suste rota ise f} H if i c i i 
From (8.9-13) the average system operating noise figure is Bef A filter is called gaussian Cit bas a transfer funeden 


I 
H(w) = —=—e 
" J 21a Woms 


where W,,,, is the root-mean-squared (rms) bandwidth, 

(a) Sketch H(w). ; 

(b) How is W,,, related to the 3-dB bandwidth? 
8-5 Two systems have transfer functions H,(w) and H,(w). 

(a) Show that the transfer function H(w) of the cascade of the (wa, which 
means that the output of the first feeds the input of the second system, is H(w) = 
H (a) FH 4(). 


OW met 


Ty T, 
Fo! +b N+ (8.9-26) 


Example 8.9-1 An antenna with temperature T, = 150 K is connected to a 
receiver by means of a waveguide that is at a physical temperature of 280 K 
and has a loss of 1.5 (1.76 dB).t The receiver has a noise bandwidth of 
W,/2n = 10° Hz and an average effective input noise (temperature T, = 


700 K. We determine the system's noise temperature T;,,, its operating 
average noise figure F, 


posse 
Mea 


ahi 


ESAS 


~ oan 


pares 


spar ns RP MER BE: 


orem wae aorae gene 


, and its available output noise power when 
Galto) = 10'? (120 dB), 
From (8.9-24) 


Ty. = 150 + 280(1.5 — 1) + 700(1.5) = 1340 K 
From (8.9-26) 


280 700 
Po=t4+4 5s Plast aS 
op TITUS) + (1S) 28.93 or 9.51 dB 


Finally, we use (8,9-23) to find N,,: 


6 
Ny. = 1,38(10~?°)1340,0(10!2) = = 12.3 mW 


—_—_—_———— 


PROBLEMS 


8-1 A signal x(t) = u(t) exp (—at) is applied to a network haying an impulse 
response h(t) = Wut) exp (—W1), Here a and Ware real positive constants and 
u(+) is the unit-step function. Find the system's response by use of (8,1-10). 

8-2 Work Problem 8-1 by using (8.1-11) to find the spectrum Y(w) of the re- 
sponse. : 


8-3 A rectangular pulse of amplitude A and duration T, defined by 
A 
xi) O<t<7 
0 elsewhere 
is applied to the system of Problem 8-1. 


(a) Find the time response yi). 
(b) Sketch your response for W = n/T and W = 2n/T. 


t A number L expressed in decibels (UB), denoted Lyy, is related to L as a numeric (power ratio) 
by Lyy = 10 log, ofL). 


(b) For a cascade of N systems with transfer functions 1/,(w), n = 1,2, ..., N, 
show that : 


N 
HMw) = T] H,(@) 
i) 


*8.6 Work Problem 8-1 if the output of the given network is applied to a second 


identical network and the response is taken from the second network. 
8-7 The impulse response of a system is 


pew? ‘O<t 
Mo= 1p pa 


By use of (8.1-8) or (8.1-10), find the response of the network to the pulse 
; A O<1<T 
a) 0 elsewhere 


where A and T are real positive constants. 
8-8 Work Problem 8-7 if the network’s impulse response is 
Pew! O<t 
} = 
Mt) e <0 
8-9 Given the network shown in Figure P8-9, 
(a) Find the impulse response f(t). 
(b) By Fourier transforming f(t), find H(c). 
(c) Sketeh A(t) and L{en). 
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stwork 
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Figure P8-9 [Reproduced from Peebles (1976), with permission of publishers Addison-Wesley, Advanced 
Book Program.) 


242 PROBAMILITY, RANDOM VARIABLES, AND RANDOM SIGNAL. PRENCIPLES 


8-10 Find the transfer function of the network of Figure P8-9 by usc of (81-13). 
HIE By using (8.1-13), find the transfer function of the network illustrated in 
Figure P&-11. Assume that no loading is present duc to any output circuilry. 


R 
i 
Uingvut C Output 
Qn rn Be Vigure PH-11 


8-12 Work Problem 8-11 for the network of Figure P8-12, 


[on 
KR Cy Output 


*8.-13 (a) Work Problem 8-i1 for the network of Figure P8-13. 
(b) Under what conditions will the network behave approximately as a 
lowpass filler? 
(c) Find a relationship between Ry, Cy, Ry, and Cy such that the network 
behaves at all frequencics as a purc resistive altcnuator, 


' Figure P8-12 


c, 


| {- does 


lnput Ry C, Output 


Figure P8-13 [Reproduced feam Peebles 

(1976), with permission of publishers Addison- 

oOo OO Wesley, Advanced Book Program.) 

8-14 Given the network shown in Figure P8-14. 
(a) If the output causes no loading on the network, find the transfer function 


H(). ; 
(hb) Define wo = !/ /LC and Qo = R/woL. Plot | H(w)|? as a function of 
X = (9 — Wg)Qo/t2o for Qo large and jw near Wo. (Hint: Use the approximation 


iw & Wy for the most significant values of w when Qo is large.) 


R 
——O 


and L Output 


eeewe eer 


Figure PR-14 
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*8.15 (a) Find the transfer function H(w) for the network shown in Figure P&-15. 
(b) Define Mm, = JC and Qg = He(R + RC and assume Qy P 1, so 
that the values of w for which H(w) is significant correspond to @ % Wa. Use 
these facts to obtain an approximation for H(c). 
(c) [fan impulse is applied to the network, find an expression for the approx- 
imate energy absorbed by R,,. (Hint: Use Parseval's (heorem). 


L R Cc 
STSCI VV 
tnput vin Ry Outpt ray 
——_———___—___+-—__9 Sipure PR-1S 


8-16°A class of filters called Butterworth filters has a power transfer function 
defined by 
| 
2 ee ee 
[Hla P= 5 wl) 
where n = 1, 2,..., is a number related to the number of circuit elements and WW 
is the 3-dB bandwidth in radians per second. Sketch | (a)? for n= 1, 2, 4, and 8 
and note the behavior. As n=» 09, what docs | H(i)? become? 
8-17 Determine which of the following impulse responses do not correspond to a 
system that is stable, or realizable, or both, and state why. 
(a) h(t) = u(t + 3) 
(b) Att) = u(ge7? 
(c) h(t) = ef sin (1) (Ug a real constant 
(d) A(t) = ude “sin (09 0) M» areal constant. 
8-18 Use (8.1-10) and prove (8.1-15). 
8-19 Show that (8.1-16) must be true if a linear time-invariant system is to be 
stable. 
8-20 A system is defined by 


X= | x(é) dé 


for all x(t) for which the integral exists. Show that the system is linear, time- 
invariant, and causal. 
8-21 A random process 

X(t) = A sin (Mt + 9) 
where A and (vg are real positive constants and © is a random variable uniformly 
distributed on the interval (—2, 7), is applicd to the network of Problem 8-1. 
Find an expression for (he network's response process using (8.2-3). 
8-22 Work Problem 8-21 fora network with impulse response 


A(t) = (te! 


2% 
5 
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4 
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8-23 A random process X(t) is applied to a linear time-invariant system. A re- 
sponse Y(t) = X(t) — X(t — t) occurs where t is a real constant. 

(a) Sketch a block diagram of the system. 

(b) Find the system’s transfer function. 
8-24 Work Problem 8-23 if the response is 


Y(t) = X(t — 1) + ["xe — &) dé 


iat 
where f, and f, are real constants. 
8-25 A random process X(t) has an autocorrelation function 


Ryx(t) = A? + BeW"! 


where A and B are positive constants. Find the mean value of the response of a 
system having an impulse response : 


ew O<t 
A(t) = 
Kt) ‘6 t<0 


where W is a real positive constant, for which X(t) is its input. 
8-26 Work Problem 8-25 for the system for which 


we" Oct 
A(t) = 
w is t<0 


8-27 Work Problem 8-25 for the system for which 


w-Wi gs 
Mi) = ‘ sin(wot) O<t 
0 t<0 


where W and ay are real positive constants. 
8-28 White noise with power density 5 W/Hz is applied to the system of Problem 
8-25. Find the mean-squared value of the response using (8.2-7). 
8-29 Work Problem 8-28 for the system of Problem 8-26. 
8-30 Work Problem 8-28 for the system of Problem 8-27. 
8-31 Let jointly wide-sense stationary processes X ,(t) and X(t) cause responses 
Y,(t) and Y(t), respectively, from a linear time-invariant system with impulse 
response h(t). If the sum X() = X,(¢) + X2(0) is applied, the response is Y(t). Find 
expressions, in terms of f(t) and characteristics of X ,(¢)and X(t), for 

(a) ELY] (6) Ryylt, ¢ + 1) 
8-32 Show that the cross-correlation function for the output com 

; rH [o} ts Y, 

and Y,(f) in Problem 8-31 is given by a 


Ry yt, 6+) = | | Ry x(t + u — v)h(ujh(v) du dv 
-wo J-w 


7 Ry,y,(t) 
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8-33 Two separate systems have impulse responses /1,(¢) and f(t). A process 
X,(t) is applied to the first system and its response is Y,(t). Similarly, a process 
X(t) invokes a response Y,(t) from the second system. Find the cross-correlation 
function of Y,(t) and Y(t) in terms of h,(t), h,(8), and the cross-correlation func- 
tion of X,( and X,(f). Assume X ,(t)and X(t) are jointly wide-sense stationary. 
8-34 Two systems are cascaded. A random process X(t) is applied to the input of 
the first system that has impulse response h,(d); its response W(t) is the input to 
the second system having impulse response h,(). The second system's output is 
Y(0). Find the cross-correlation function of W(t) and Y(t) in terms of Ay(q and 
h(t), and the autocorrelation function of X(t) if X(t) is wide-sensc stationary. 
8-35 Let the two systems of Problem 8-34 be identical, each with the impulse 
response given in Problem 8-26. If ELX(t)] = 2and W = 3 rad/s, find ECL Y(t]. 
8-36 The random process X(t) of Problem 8-21 (the signal) is added to white 
noise with power density . 79/2, where .W% is a positive constant, and the sum is 
applied to the network of Example Bel 

(a) Find the power spectrums of the oulput signal and output noise. 

(b) Find the ratio of output signal average power to oulpul noise average 
power. 

(c) What value of W = R/L will maximize the ratio of part (b)? 


8-37 A random process X(t) having autocorrelation function 
Ry x(t) = Pew 


where P and a are real positive constants, is applied to the input of a system with 


impulse response 

wee” 0 <t 

h(t) = { ° 

0 <0 
where W is a real positive constant. Find the autocorrelation function of the 
network’s response Y(t). 
8-38 Find the cross-corrclation function Ryy(t) for Problem 8-37. 
8-39 For the processes and system of Problem 8-31, show that the power spec- 
trum of Y(t) is 

Syy(e) = | H(@) PLS x.x.(@) + Sxax (©) + Sxyx eo) + Syyx(@e)] 
8-40 If X,() and X,(/) are statistically independent random processes in 
Problem 8-31, use the results of Problem 8-39 to show that the oulput power 
spectrum becomes 
Syy(w) = | AH) PES x, x, (2) + 8x,x,{c) + 4nX ,X, 6(o)] 

8-41 Rework Example 8.4-1 when the network is replaced by two identical net- 
works in cascade, that is, when H(w) = [1 + (jwL/R)\~?. 
8-42 Show that (8.4-7) and (8.4-8) are (rue. 


ood 


— 
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) = jw is a differentiator, its input is 


8-43 A network with transfer function H(cw ¢ 
and its outputis X() = AX (fat. 


the wide-sense stationary random process X(!) 


(a) By using (8.4-7), show that 
\ 


IR y y(t 
Ryalt) = east 


dt 


(bh) By using (8.41), Show that 
Ry xt) 


Realt) = — a 
8-44 Given the random process 
rT +T 
= X(é) d 
rong | xOu 


where X(1) is a wide-sense stationary process. Use (8.2-1) to show that the power 


spectrum of Y(t) is 


sin (wT) ? i 
Syy(@) = Syxto SO | 


8-45 Prove (8.5-6). 


8-46 A random process X(t) has a power spectrum Sxx(w) that is nonzero only 
for —Wy <a < Wy, where Wy, is a real positive constant. X(t) is applied to a 
system with transfer function 


ca) = 1 + o/Wu) —Wy <a < Ws 


rk's response Y(t) in terms of the rims 


Find the average power Pyy in the netwo 
and the average power Pyy in X(t). 


bandwidth of Sxx(w), the constant W,,, 
Discuss the effect of letting Wy 0. 


8-47 Find the noise bandwidth of the system having the power transfer function 


1 
| H(w)? = Tele Tory 


where H’ is a real positive constant. 


8-48 Work Problem 8-47 for the function 


i 
Mol = awe? 
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8-49 Work Problem 8-47 for the function 


| 
8 Sy 
HON = Troe 


8-50 White noise with power density MV o/2 is applicd to a lowpass network for 
which | //(0)| = 2; it has a noise bandwidth of 2 MHz. If the average output noise 
power is 0.1W ina 1-Q resistor, what is “9? 

8-51 While noise with power density AM, [2 is applied to an ideal lowpass filter 
with bandwidth W. 

(a) Find and sketch the autocorrelation function of the response. 

(b) If samples of the output noise taken at times t, = an/W, n =, +i, 
+2,..., are considered as values of random variables, what can you say about 
these random variables? 

8-52 Work Problem 8-51 for an ideal bandpass filter centered on a frequency 
(vo/2n that has a bandwidth W. Assume sample times are now {, = nan/W,n = 0, 
+14, 42,.... 
*3.53 A band-limited random process N(1) has the power densily spectrum 


—-W/2<w— Wo s W/2 
—~W/2<s wt Wo s W/2 


P cos [n(w — @)/W] 
Syn(w) = 4 P cos (x(w + Wo)/W) 


0 elsewhere 


where P, W, and wy > W are real positive constants. 

(a) Find the power in N(0). 

(b) Find the power spectrum Sxx(w) of X(t) when N(t) is represented by 
(8.6-2). 

(c) Find the cross-correlation function R yy(t). 

(d) ‘Are X(t) and Y() orthogonal processes? 

*8.54 A band-limited random process is given by (8.6-2) and has the power 

density spectrum shown in Figure P8-54. 

(a) Sketch Syx(w). 

(b) Sketch Syy(w), ifa sketch is possible. 
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Figure P8-54 
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*8-55 Work Problem 8-54 for the power spectrum of Figure P8-55. 


Swalw) 


Figure P8-55 


*8-56 Use (8.6-2) and derive (8.6-23), 


8-57 A sonar echo system on a submarine transmits a random noise n(t) to deter- 
mine the distance to another “target” submarine. Distance R is given by vr,/2 
where v is the speed of the sound waves in water and tp is the time it takes the 
reflected version of n(t) to return. Its block diagram is shown in Figure P8-57. 
Assume that n(t) is a sample function of an ergodic random process N(t) and T is 
very large. 

(a) Find V in terms of a correlation function of N(2), 

(b) What value of the delay t; will cause V to be maximum? 

(c) State in words how the submarine can determine the distance to the 
target. 


Round: es delay = ry 


Transmitter 


cf, | 
noon 


Distance 


i 
ap lo ()di 


Figure P8-57 
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8-58 ‘Two resistors with resistances R, and R, are connected in parallel and hive 
physical temperatures 7) and 73, respectively, 

(a) Find the effective noise temperature J, of an equivalent resistor with 
resistance equal to the parallel combination of R, and R,. 

(b) WT, = Tz = T, what is 7}? 
8-59 Work Problem 8-58 for three resistances R,, R2, and Ry in parallel when 
they have physical temperatures 7), 7,, and Ty, respectively. 
8-60 Work Example 8.7-2 if a second capacitor is placed across the resistince 
R,. Is it possible to choose C, so that 7, is independent of frequency.? 


*8-61 Find the effective noise temperature of the network of Figure P8-61 if R, 


and R, are at physical temperatures 7, and 73, respectively, 


Lt 


Figure P8-61 


8-62 A two-port network is illustrated in Figure P8-62. Find its available power 
gain. 


Temperature = Ty 


1 Noisy two-part i 
(pate _ Figure P8-62 


$-63 If the two-port network of Problem 8-62 has a physical temperature 7), and 

is driven by a source of resistance R, and effective noise temperatiire T,, what is 

the effective inpul noise temperature of the network? 

8-64 If the output of the network of Problem 8-62 is connected to the input of a 

sae indentical network, what is the available power gain of the cascade if 
=50,R,=3 Nand ky = 70? 

; 65 Show that (8.8-4) is valid. 

8-66 In a cascade of M network stages for which the mth stage has available 

power gain G,, and operating spot noise figure F,,,, when driven by all previous 

stages as its source, show that the overall cascade’s operating spot noise figure is 


F.=F 5 GlFop2 = 1) safe Tran = 1) Fopat 1 1) 
0: opt + TG, TGC, 3 Gis 


where Ty,-4) is (he temperature of all stages prior to stage m treated as a source. 
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8-67 An amplifier has a standard spot noise figure Fo = 6.31 (8.0 dB). An engi- 
neer uses the amplifier to amplify the output of an antenna that is known to have 
an antenna temperature of T, = 180 K. 

(a) What is the effective input noise temperature of the amplifier? 

(b) What is the operating spot noise figure? 
8-68 In a cascade of M stages for which Fons = 1, 2,206. M, is the standard 
spot noise figure of stage m which has available power gain G,,, show that the 
standard spat noise figure of the cascade of networks is 


Fo—1 , Fox! fc tl 
Fo=Fout ae 4A EG Gy 
1 ry arg 1,G) Iaet 


ages for which 7,, = 200 K (first stage), Ta = 


8-69 An amplifier has three st 
ain of the second 


480 K, and T,; = 1000 K (last stage). If the available power 
stage is 5, what gain must the first stage have to guarantee an effective input 
noise temperature of 250 K? 

8-70 An amptifier has an operating spol noise figure of 10 dB when driven by a * 
source of efiective noise temperature 225 K. 

(a) What is the standard spot noise figure of the amplifier? 

(b) Ifa matched attenuator with a loss of 3.2 dB is placed between the source 
and amplificr’s input, what is the operating spot noise figure of the altenualor- 
amplifier cascade if the allenuator’s physical temperature is 290 K? 

(c) What is the standard spot noise figure of the cascade in (b)? 

8-71 One manufacturer sells a microwave receiver having an operating spot 
noise figure of 10 dB when driven by a source with effective noise temperature 
130 K, Another sells a receiver with a standard spot noise figure of 6 dB. 

(a) Find the elfective inpul noise temperatures of the two reccivers. 

(b) All other parameters, such as gain, cost, cle, being the same, which 
reeciver would be the best to purchase? 

8-72 What is the maximum average effective input noise temperature that an 
amplifier can have if ils average standard noise figure is to not execed 1.7? 


8-73 An amplifier has an average standard noise figure of 2.0 dB and an average 


operating noise figure of 6.5 dB when used with a source of average elYective 


source temperature T,. What is Ty) 
8-74 An antenna with average noise temperature 60 K connects to a receiver 
through various microwave elements that can be modeled as an impedance- 
matched allenuator with an overall loss of 2.4 dB and a physical temperature of 
275 K. The overall system noise temperature is T.,, = 820 K. 

(a) What is the average effective input noise temperature of the receiver? 

(b) What is the average operating noise figure of the attenuator-receiver 
cascade? 

(c) What is the available output noise power of the receiver if 
able power gain of 110 dB and a noise bandwidth of 10 MHz? 
of Problem 8-74 is considered as a noise 


t has an avail- 


8-75 Mf the antenna-atienuator cascade 
source, What is its average effective noise temperature? 
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8-76 The loss L in Figure 8.9-la is replaced by two cascaded matched attenu- 
ators, one with loss L, at temperature 7, attached to the antenna output, and 
one with loss L, at temperature T, that connects to the receiver. Derive A new 
expression for T,,, analogous to (8.9-24). 


ADDITIONAL PROBLEMS 


8-77 A network is driven by a resistive source as shown in Figure P8-77. Find: 
(a) Z,, (b) Z,, and (c) G,. (d) ts the network a matched attenuator? 


Source 


R= WOR Z, Z 
Figure P8-77 . 


8-78 A network has the transfer function 
Qelnl20 
* (20 + joo)? 
(a) Determine and sketch ils impulse response. (Tint: Use Appendix 12.) 


(b) Is the network physically realizable? 
(c) Determine if the network is stable by evaluating J in (R.1-16). 


H(w) 


P , 
8 79 Show that the impulse response of a cascade of N identical networks, cacl 
with transfer function cas 


H,(w) = 1a + im) : 
where « > Ois a constant, is given by 
k | 1% =i 
h(t) = u(t) ——— - 
v(t) = u(t) We =| exp (—at) 
8-80 A signal 
x(t) = u(t) exp (—af) 
is applied to a network having an impulse response 
h(t) = u(Qw?t exp (— Wo 


Here a> 0 and W > 0 are real constants B x 
ian . By use of (8.2-2) find the network's 
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8-81 Work Problem 8-80 assuming 
h(t) = ut W3e? exp (~W) 


8-82 A stationary random process X(t) is applied to the input of a system for 
which 


A(t) = 3u(t? exp (—88) 
If E[X(1)] = 2 what is the mean value of the system’s response Y(t)? 


8-83 Work Problem 8-28 for the system of Problem 8-82, 


8-84 White noise with power density 9/2 js applied to a network with impulse 
response 


A(t) = u(QWt exp (—Wo) 


where W > 0 is a constant, Find the cross-correlations of the input and output. 
8-85 Work Problem 8-84 for a network with impulse response. 


h(t) = u(t)We sin (Wot) exp (— Wa) 
where wo is a constant. 
8-86 A random process X(t) is applied to a network with impulse response 


A(t) = u(t exp (—bd 


where b > 0 is a constant. The cross-correlation of X(t) with the output Y(t) is 
known to have the same form: 


Ryyl(t) = u(t)t exp (— br) 
(a) Find the autocorrelation of Y(¢), 
(b) What is the average power in Y(t)? 
8-87 Work Problem 8-86 except assume 
h(t) == w(t)t? exp (— bt) 
and 
Ryy(t) = u(t)t? exp (— br) 
8-88 Two identical networks are cascaded, Each has impulse response 
A(t) = u(t)3t exp (— 48) 


A wide-sense stationary process X(t) is applied to the cascade’s inpul. 
(a) Find an expression for the response Y(t) of the cascade. 
(b) If ELX(t)) = X = 6, find ELY(c)]. 


8-89 A stationary random process X(¢), having an autocorrelation function 
Ryx(t) = 2 exp (—4]7]) 


is applied to the network of Figure P8-89. Find: (a) Syx(w), (b) | H(w)/?, and 
(¢) Syy(). 
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(,a4F 


Figure P8-89 


8-90 A wide-sense stationary process X(t), with mean value 5 and power spec- 
trum 


Syx() = 5025(w) + 3/E1 + (@/2)?] 
is applied to a network with impulse response 
h(t) = 4 exp (—4}4]) 

(a) Find H(w) for the network, 

Determine: (b) the mean ¥, and (c) the power spectrum of the response Y(t). 
8-91 White noise, for which Ryx(t) = 107 7d(z), is applied to a network with 
impulse response 

A(t) = u(t)3¢ exp (— 48) 
(a) Use (8.2-9) to obtain the network’s output noise power (in a f-ohm 


resistor), 
(b) Obtain an expression for the output power spectrum. 


8-92 White noise with power density 9/2 = 6(1075) W/Hz is applied to an 
ideal filler (gain = 1) with bandwidth W (rad/s). Find W so that the output's 
average noise power is 15 watts. 

8-93 An ideal filler with a midband power gain of 8 and bandwidth of 4 rad/s 
has noise V(f) at its input with power spectrum 


Sxy(to) = (50/,/8n) exp (—c0/8) 


What is the noise power at the network’s outpul? 

8-94 While noise with power densily 9/2, Wo > 0 a constant, is applied to a 
lowpass network for which H(0) = 2 and its noise bandwidth is 2 MHz. If 
average oulpul noise power is 0.1 W ina l-ohm resistor, what is “9? 


8-95 A system’s power transfer function is 


| H(co) |? = 16/[256 + w*] 


| 


*8-96 Assume a band-limited random process 
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(a) What is its noise bandwidth? 
(b) If white noise with power density 6(107 
find the noise power in the system's output. 


4) W/Hz is applied to the input, 


N(t) has a power spectrum 


Sxl) = 
Bula — Wo + Hy) - (co — Wo — W4)} exp [—-al — 9 + WY) 


4+ BE — a — oy + W4) — ul — M9 7 1V,)} exp [—a(- 9 = Hg + Wy) 


where B, ay, H4, and IV, are positive constants, and ais a constant. 

Assume 209 > I, + HY, and find analytical expressions for (a) the power 
spectrum Syy(«) and (b) the cross-power spectrum Syy(ca) for the processes X(t) 
and Y(1) involved in the representation of (8.6-2) for N(O). 

(c) Sketch Sy y(en) and Syy(en) for WY, = W,/2and a = 1/4. 

(d) Repeat part (c) except with a = —1/M,. 

#8.97 Find the functions Ry x(t) and Ryy(t) applicable in Problem 8-96. 
8-98 Determine the elfective noisc temperature of the network of Figure P8-98 if 
resistors R, and R, are at different physical temperatures T, and T; , respectively. 


Figure P8-98 


8-99 Two resistors in series have different physical temperatures as in Example 
8.7-1. Let Ry and R, be independent random variables uniformly distributed on 
(1000, 1500) and (2200, 2700), respectively. Their average resistances arc (hen 
R, = 1250 Q and R, = 24509. 
(a) What is the ellective noise temperature oO 
T, = 250 K and T, = 330 K and average resistors arc uscd? 
(b) What is the mean effective noise temperature of the source for the same 


values of 7, and 7? ; 
8-100 Anamplificr has three slages for which T., = 150 K (first stage), Tea = 350 K, 
and Tes 600 K (output stage). Available power gain of the first stage is 10 and 
overall input cffective noise temperature is 190 K. 

(a) What is the available power gain of the second stage? 

(b) What is the cascade’s standard spot noise figure? 

(c) What is the cascade’s operating spot noise figure when used with a source 
of noise temperature 7, = 50 K? 
R-101 Three networks are cascaded. Avail 
stage), G, = 6, and G, = 20 (outpul stage). Respective inpu 
= 100K, and T,, = 280K. 


f the two resistors as a sourc[e if 


able power gains arc G, = 8 (input 
t effective spot noise 


temperatures are rT, =40K, Ty 
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(a) What is the input effective spot noise temperature of the cascade? 

(b) If the cascade is used with a source of noise temperature T, = 30 K, find 
the percentage of total available output noise power (in a band dw) due to cach 
of the following: (1) source, and the excess noises of (2) network 1, (3) network 2, 
and (4) network 3. 

8-102 An antenna with effective noise temperature 7, = 90 K is connected to an 
attenuator that is ata physical temperature of 270 K and has a loss of 1.9, What 
is the effective spot noise temperature of the antenna-attenuator cascade if its 
outpul is considered as a noise source? 

8-103 An amplifier, when used with a source of average noise (emperalure 60 K, 
has an average operating noise figure of 5. 

(a) What is 7? 

(b) If the amplifier is sold to the engincering public, what noise figure would 
be quoted in a catalog (give a numerical answer)? 

(c) What average operating noise figure results when the amplifier is used 
with an antenna of temperature 30 K? 

8-104 An engineer purchases an amplifier with average operating noise figure of 
1.8 when used with a 50-Q broadband source having average source temperature 
of 80 K. When used with a different 50-Q source the average operating noise 
figure is 1.25. What is the average noise (emperature of the source? 
8-105 An amplifier with a noise bandwidth of at least 1.8 MHz is needed by an 
engineer. Two units from which he can choose are: unit !—average standard 
noise figure = 3.98, noise bandwidth = 2.0 MHz, and available power 
gain = 106; unit 2—average standard noise figure = 2.82, noise bandwidth = 
2.9 MHz, and available power gain = 10°. 
Find: (a) T, for unit 1, (b) T, for unit 2, (c) excess noise power of unit 1, and 
(il) excess noise power of unit 2. 
(ec) If the source’s noise temperature T. is very small, which unit is the best lo 
purchase and why? 
(f) 1f T, > T,, which is best and why? 
*8.106 A resistor is cooled to 75 K and serves as a noise source for a network 


with available power gain 
G,(w) = 10°4/(10% + «?)* 


(a) Write an expression for the power spectrum of the network's output noise 


that is duc to the source. 

(b) Compute the available output noise power that is duc to the source 
alone. 
8-107 A broadband antenna, for which T, = 120 K, connects through an altenu- 
ator with loss 2.5 to a receiver wilh average input effective noise temperature 
80 K, available power gain 10'2, and noise bandwidth 20 MHz. The antenna 
and attenuator both have a physical (emperature of 200 K. 

(a) What is the attenuator’s input effective noise temperature? 


(b) What is the system's noise temperature? 


| 
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(c) Find the average standard noise figure of the receiver by itself. 

(¢) What is the available noise power at the recciver's output (in system 
Operation)? 

(e) Determine the input effective noise temperature of the attenuator-receiver 
taken as a unit. 

(f) What is the average operating noise figure of this system when the 
antenna is the source? 

8-108 An antenna with average noise temperature 120 K connects to a receiver 
(through an impedance-matched attenuator having a loss of 1.5 and physical tem- 
perature 75 K. For the overall system 7,,, = 500 K. 

(a) What is the average effective input noise temperature of the receiver? 

(b) What is the average operating noise figure of the attenuator-recciver 
cascade? 

(c) What is the available output noise power of the receiver if its available 
power gain is 120 dB and its noise bandwidth is 20 MHz (system is connected)? 
8-109 A receiving system consists of an antenna with noise temperature 80 K 
that feeds a matched attenuator with physical lemperature 220 K and loss 2.6. 
The attenuator drives an amplifier with average effective noise temperature !70 
K, noise bandwidth 4 MHz, and available power gain 10°, 

Find: (a) the overall system's average noise temperature Tie (b) the available 
noise power N,, at the system's output, (c) the total noise power available al the 
atlenuator’s output (within the noise bandwidth) and how much of the total (as a 


percentage) is duc to the antenna alone, and (d) the average operating noise figure 
F, of the system. 


CHAPTER 


NINE 
OPTIMUM LINEAR SYSTEMS 


9.0 INTRODUCTION 


The developments of the preceding chapter related entirely to the analysis of a 
linear system. In this chapter we do an about-face and concentrate only on the 
synthesis of a linear system, In particular, we choose the system in such a way 
that it satisfies certain rules that make it optimum. 

In designing any optimum system we must consider three things: input speci- 
fication, system constraints, and criterion of optimality. 

Input specification means that al least some knowledge must be available 
about the input to the system, For example, we might specify the input to consist 
of the sum of a random signal and a noise. Alternatively, (he input could be the 
sum of a deterministic signal and a noise. In addition, we may-be able to specify 
signal and noise correlation functions, power spectrums, or probability densities. 
Thus, we may know a great deal about the inputs in some cases or little in others. 
Regardless, however, there is some minimum knowledge required of the charac- 
teristics of the input for any given problem. 

System constraints define the form of the resulting system. For example, we 
might allow the system to be linear, nonlinear, time-invariant, realizable, etc. In 
our work we shall be exclusively concerned with linear time-invariant systems 
bul will not necessarily require that they be realizable. By relaxing the realizabil- 
ily constraint, we shall be able to introduce the most important topics of interest 
without undue mathematical complexily, 

In principle, there is great latitude available in choosing the criterion of opti- 
matity. Ina practical sense, however, it should be a meaningful measure of 
“poodness” for the problem at hand and should correspond to equations Chat 


257 


tone 


ee 


BEE 
aera pre Anaeod 


| 
mot FF 


pr 


288 PRORABILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 


are miuthematically tractable, We shall be concerned with only two criteria. One 
will involve the minimization of a suitably defined error quantity, The other will 
relate to maximization of the ratio of a signal power to a noise power. This last 
criterion leads us to an optimum system often called a atatched filter. 


9.1 SYSTEMS THAY MAXIMIZE SIGNAL-TO-NOISE RATIO 


An important class of sysicms involves the transmission of a deterministic signal 
of known form in noise. A digital communication system is onc examplef where, 
during a time interval 7) a known signal may arrive at the receiver in the pres- 
ence of additive noise. The presence of the signal corresponds to transmission of 
a digital “ 1," while absence of the signal occurs when a digital “0” is (ransmitted 
(noise is always present). It would scem reasonable that some system (or filtert) 
could be found that would enhance its output signal power at some instant in 
ime while reducing its output average noise power. Indeed, such a filter that 
maximizes this output signal-to-noise ratio can be found and itis called a matched 
filter, It can be shown that decisions made as to whether the signal was present 
or not during time interval T have the smallest probability of being in error if 
they are based on samples taken at the times of maximum signal-to-noise ratio, 
Although our comments here are directed toward a digital communication 
system, we shall find as we progress that the matched filter concept is a broad 
one, applying to many situations. 

In this section we shall consider the optimization of a lincar time-invariant 
system when (he input consists of the sum of a Fouricr-transformable determin- 
istic signal x(f) of known farm and continuous noise (1). If we denote by x,(f) and 
(ft) the output signal and noise, the criterion of optimality we choose is the max- 
mization of the ratio of the oulpul signal power at some time ¢, Fo the output 
average noise powcr. Thus, with 1,(¢) assumed to be a sample function of a wide- 
sense stationary random process§ N,(1), we maximizc 


S\ _ Latte)? 
| = al Ae! 
(i) ELN2(0) Den) 
where 

§, = 1 xolta) |? (9.1-2) 


is the output signal power at time ¢£, and 
Ny = FENG] (9.1-3) 


is the oulpul average noise power. 


example, many other systems such as radars, sonars, radio 
automobile crash avoidance systems are other examples. 
t We offen use the words system, filler, of network in this chapter to convey the sane meaning. 
is equivalent to assuming the input noise is from a wide-sense stationary 


§ This assumplion i 
random process since the system is assumed fo be linear and time-invariant (see Section 8.2). 


ft Although we discuss only this 
altimeters, ionospheric saunders, and 
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Matched Filter for Colored Noise 


Define X(m) as the Fourier transform of x(f), and [H{im) as the transfer function of 
the system. The output signal at any time ¢ is 


1 oO 
x,(t) = a | X(w)H(w)e" deo (9,1-4) 
From (8.4-6), the output average noise power can be written in the form 


1(™ 

N, = ELN2()] = on | Spal) | Ma)? dw (9.1-5) 
where Syy(w) is the power density spectrum of the random process, denoted N(t), 
that represents the input noise n(¢). By use of (9.1-4) at time f, and (9.1-5), we can 
write (9.1-1) as 


1 {° : 
(3) 2 an [7 xeon din 
cara 2 (9.1-6) 
Qn |}. Sun) | H(w) ? do 


To find H(w) that maximizes (9.1-6), we shall apply the Schwarzt inequality. 
If Aten} and B(w) are two possibly complex functions of the real variable cw, the 
inequality states that ' 


2 C) ro) 
< | | Aco) |? de» { { Blew) |? dew (9.1-7) 


i A(w)B(w) dw 


The equality holds only when B(w) is proportional to the complex conjugate of 
A(on); that is, when 


B(w) = CA*(w) (9.1-8) 


where C is any arbitrary real constant. 
By making the substitutions 


A(w) = JS Synler) Ho) (9.1-9) 


X(w)el™"* 


Bi 2 9.1- 
1 Bunt ee 
in (9.1-7) we obtain 
2 i ere ea 1 f* [XP 
Fe I _KlaHteye! i < i _Sun(o)| H(w)|? dw On ie “Saalto) dw 
(9.1-11) 


+ Named for the German mathematician Hermann Amandus Schwarz (1843-1921). 
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With this last result, we wrile (9.1-6) as 


S.\ 1 f? LX@)P 
(it) 55 Set oo 


The maximum value of (§,/N,) occurs when the equality holds in (9.1-12), which 
implies that (9.1-8) is (rue, Denote the optimum filter transfer function by Hye). 
We find this function by solving (9.1-8) using (9.1-9) and (9.1-10); the resull is 
* 
Hyp (to) = = cel 
nC Syy(w) 

From (9.1-13), we find (hat the optimum filter is proportional to the complex 
conjugate of the inpul signal's spectrum; we might say that the system is there- 
fore matched to the specified signal since it depends so intimately on it. Hy (@) is 

ilso inversely proportional to the power spectrum of the input noise. In general, 
this noise has been assumed nonwhite; that is, colored. Because of these fuets, an 
optimum filter given by (9.1-13) is called a matched filter for colored noise. 

H,,(w) is also proportional to the inverse of the arbitrary constant C. In 
other words, H, (cw) has an arbitrary absolute magnitude. This fact allows the 
optimum system to have arbitrary gain. Intuitively, we feel that this should be 
true because gain affects both input signal and input noise in the same way, and, 
in the ratio of (9.1-1), gain cancels. 

The time ¢, at which the output ratio (5,/N,) is maximum enters into the 
optimum system transfer function only through the factor exp (—jwr,). Such a 
factor only represents an ideal delay. Since ¢, is a parameter that a designer may 
have some latitude in choosing, its value may be selected in some cases to muke 
the optimum filter causal. 

In general, the system defined by (9.1-13) may not be realizable. For certain 
forms of colored noise realizable filters may be found (Thomas, 1969, Chapter 5). 
In practice, one can always approximate (9.1-13) by a suitably chosen real filter. 


edule (9.1-13) 


Matched Filter for White Noise 


Ifthe input noise is white with power density 47 9/2, the optimum filter of (9.1-13) 
becomes 


Hyp (to) = KX*(w)e 7M" , (9.1-14) 


where K = I/nC. {79 is an arbitrary constant. Here the optimum filter is related 


only to the input signal’s spectrum and the time that ($,/N,) is maximum. Thus,. ” 


the name matched filter is very appropriate. Indeed, the name was originally 
attached to the filter in white noise; we have liberalized the name to include the 
preceding colored noise case. 

The impulse response denoted hyw(t) of the optimum filler is the inverse 
Fourier tranform of H,,,(w). From (9.1-14), it is easily found that 


Irglt) = Kx*(t, = t) (9,1-15) 


' 
» 
+ 


i 
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For real signals x(1), (9.1-15) reduces to 

Foolt) = Kx(t, — 9 (9.1-16) 
Equation (95-16) indicates that the impulse response is equal to the input signitl 
displaced to a new origin at f= 1, and folded about this point so as to “run 
backward.” 


me I tm 


Example 91-1 We shall find the matched filler for the signal of Migure Yet 
when received in white noise. From (9.1-16), the matched: filler’s impulse 
response is as shown in (b). By Fourier transformation of the waveform in (h), 
we readily obtain 
sin (wt/2) 7 delta tte 2 
(wt/2) 

An alternative development consists of Fouricr-transforming the input signal 
to get X(w) and then using (9.1-14). 

Whether or not any chance exists for the matched filler to be realizable 
may be determined from the impulse response of Figure 9.1-1b. Clearly, to be 
causal, and therefore realizable, the delay must be at least t — t,; that is 


Hyy(0o) = KAt 


[i27T-T, 
x(t) 
fF. 
“ty 0 TT, ! 
(a) 
Hon) 
KA | 
fy tty-t 0 ty ty tTo { 
(b) 


Input Integrator 


fodder Output 


a) 


Figure 9.1-1 A matched filler and its related signals, (a) Input signal, (b) the Hiter’s impulse response, 
and (c) the filler’s block diagram [Reproduced from Peebles (1976), with permission of publishers 
Addison-Wesley, Advanced Book Program.) 
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Ife assume this last condition is satisfied, the optimum filter is illustrated in 
(c) where the arbitrary constant K is sct equal to 1/A. This filter still requires 
that perfect integrators be possible. Of course, they are not. However, very 
good approximations arc possible using modern operational amplifiers with 
feedback, so for all practical purposes matched filters for rectangular pulses 
in white noise may be constructed.t 


Oe Bee 


9.2 SYSTEMS THAT MINIMIZIE MEAN-SQUARED ERROR 


A sccond class of optimum systems is concerned with causing the oulpul to bea 
good estimate of some function of the input signal which arrives along with addi- 
tive noise, One example corresponds to the output being a good estimate of the 
derivative of the input signal. In another case, the system could be designed so 
that its output is a good estimate of cither the past, present, or future value of the 
input signal. We shall concern ourselves with only this last case. The optimum 
system or filter that results is called a Wiener filter. 


Wiener Filters 


The basic problem to be studied is depicted by Figure 92-1. The input signal x(1) 
is now assumed to be random; it is therefore modeled as a sample function of a 
random process X(t). It is applied to the input of the system along with additive 
noise n(t) that is a sample function of a noise process N(f). We assume X(1) and 


t Other techniques using integrate-and-dump methods exist. See Peebles (1976), pp. 4G1-362. 
+ After Norbert Wiener (1894-1964), a great American mathematician whose work has tremen- 
dously alfveted many areas of seience and engincering. 
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Figure 92-1 Opertions that define the Wiener biter probler. 
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N(t) are jointly wide-sense stationary processes and that N(1) has zero mean. The 
sum of signal and noise is denoted W(t): 


W(t) = X() + N(D (9.2-1) 


The system is assumed to be linear and time-invariant with a real impulse 
a h(t) and a transfer function H(). The output of the system is denoted 
(1). 
In general, we shall select H(w) so that Y(t) is the best possible estimate of the 
input signal X(f) at a time f+ 4,3 that is, the best estimate of X(t + ¢,). If t, > 0, 
Y(t) is an estimate of a future value of X(1) corresponding to a prediction filter. Mf 
t, <0, Y(t) is an estimate of a past valuc of X(1) and we have a smoothing filter. Vf 
, = 0, Y() is an estimate of the current valuc of X(1). 
Now if Y(t) differs from the desired (ruc value of X(t -+f,), we make an 
crror of 
a(t) = X(t + ¢,) — Y(t) (9.2-2) 


This error is illustrated conceptually in Figure 9.2-1 by dashed lines, The 
optimum filter will be chosen so as lo minimize the mean-squared value of e(t).t 
We shall not be concerned with obtaining a system that is realizable. Some infor- 
mation is given by Thomas (1969) on the more difficult problem where H(w) must 
be realizable. Thus, we seck to find H(w) (hat minimizes 


ELe%()] = EL{X(e +t.) — Y()}7I 
= FLX%(t +t) — 2Y (OX (6 + be) + ¥7(0) 
= Ryx(0) — 2Ryx(t,) + Ryy(0) (9.2-3) 


From the Fourier transform relationship between an autocorrelation func- 
tion and a power spectrum, we have 


I nm 
R,xx(0) = im | Syxx(w) dw (9.2-4) 


where Syy(m) is (he power density spectrum of X(t). From a similar relationship 
and (8.4-1) we have 


“my 


1 
Ryy(0) = on i; Swiw(w) | H(w)|? deo (9.2-5) 


where Syyy(c) is the power spectrum of W(!). By substitution of (9.2-4) and (9.2-5) 
into (9.2-3), we have 


ao 


1 
Ele?(t)] = —2Ryx(t.) + mal (Syx(o) + Swio) | H(@) 7} dw (9.2-6) 


-o 


¢ We could elect to minimize the average error, or even force such an error to be zero, This 
approach does not prevent large positive errors from being offset by large negative errors, however. 
Ry minimizing the squared error, we eliminate such possibilities. 
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To reduce (9.2-6) further, we develop the cross-correlation function: 


Ryx(t.) = ECY(QX(t + 4)] = of xe + t,) i‘ A(Q)W(t — ¢) a] 


= [- Ruyzlte + OME) dé 9.2.7) 


-@ 


where Ry,(:) is the cross-correlation function of W(t) and X(t). After replacing 
Ry x(t, + €) by its equivalent, the inverse Fourier transform of the cross-power 
spectrum 8,y,(w), we obtain 


Ryxt,) = [" = [- Swale? dav h(E) dé 


~o 2m J. 


= [" Sia(aye} heels as} day 


as -w 


{ a 
= os | Sip (a) H(—w)e™ deo (9,2-8) 


on 


Substitution of this expression into (9.2-6) allows it to be written as 


Ele] == | {Syqlo) — 2Syyx(o)H(— coe!" + Sy yy(co)| Heo) 2} lev 
2n Jew 


(9.2-9) 

The transfer function that minimizes E{e?(t)] is now found, We may write 
H(qw) in the form 

H(w) = A(w)e/®™ (9.2-10) 


where A(cw) is the magnitude of H(w), and B(w) is its phase, Next we observe that 
Sxx(w) and Syyy(w) are real nonnegative functions, since they are power spec- 


(rums, while (he cross-power spectrum Sy (w) is complex in general and can be 
wrilten as 


Sy x(to) = C(w)e/? (9.2-11) 
After using (9.2-10) and (9.2-11) in (9,2-9) and invoking the fact that 
H(—«w) = H*(w) (9.2-12) 
for filters having a real impulse response A(t), we obtain 
: oe aa 
Efe%(1)) = = | {Sxx(@) + Swy(w)A(w)} dw 
1 w 
ar | 2C(w) A(w)elwte + Pod - Ba ay (9.2-13) 


We minimize E[e(t)] by first selecting the phase of H(w) to maximize the second 
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integral in (9.2-13) and then, with the optimum phase substituted, minimize the 
resulting expression by choice of A(w). Clearly, choosing 
Bly) = wt, + D(w) (9.2- 1-1) 


will maximize (he second integral and give the expression 


Efe?()] = a2 [" {Syx(@) — 2C(w)A(2) + Swil@)A2(w)} do 


n -wv 
Lift fs Cw) C(w) ' as HORE 
= [. {Sato - Sule) + Syyla] AC - | da) (9,2-15) 


In writing the last form of (9.2-15), we have completed the square in Aw). Finally, 
it is clear that choosing 
Cl) 

{((v) = ———— (9,2-16) 

: Siw) 
will minimize the right side of (9.2-15) By combining (9.2-16), (9.2-14), and 
(¥,2-11) with (92-10) we have the optimum filter transfer function whieh we 
denote H,,(@): 


Syey(to 
Hy) = water) 2) pile 


= (9.2-17) 
Siiv(w) 


For the special case where input signal and noise are uncorrelated, it is casy 
(o show that 
Sywir(o) = Syx(@) + Syn(er) (9.2-18) 
Sxl) = 8 (0) (9.2-19) 
where Sya(cv) is the power spectrum of N(t), Hence, for this special case 


Syl) pit 


i = Ss. 
up) Syy(ww) + Spy) 


(9.2-20) 


Example 9.2-1 We find the optimum filler for estimating V(t -F 4.) when 
there is no input noise. We fet Syy(ia) = 0 in (92-20): 


Hea) = gitite 


This expression corresponds to an ideal delay line with delay -—1,. Ht, > 0, 
corresponding to prediction, we require an unrealizable negative delay line, If 
t, < 0, corresponding to a smoothing filter, the required delay is positive and 
realizable, Of course, ft, = 0 results in H,,,(@) = 1. In other words, the 
optimum filter for estimating X(t) when no noise is present is just a direct 
connection from input (o output, a result that is intuitively agreeable. 
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Minimum Mean-Squared Error 


On substitution of (9.2-17) into (9.2-15), we readily find the mean-squared error of 
the optimum filter 


| 7 Sy lSwl) Pon [Sino ? 
i ln 


er pty =o 
Ke (asin 2 {" ru Syywler} 


nt 
For the special case where input signal and noise are uncorrelated, this equation 


(9,2-21) 


reduces to 
I 


ECan = 35 | SxxlSual) ey 


it (9.2-22) 
n Jou, Syxlev) + Syalr) 


9,3 OPTIMIZATION BY PARAMETER SELECTION 


We conclude our discussions of optimum lincar systems by briefly considering a 
second approach that minimizes mean-squared crror. The problem we undertake 
is identical to that of the last section up to (9.2-9), which defines the mean- 
squared error. Now, however, rather than seeking the filter that minimizes this 
error, we specify the form of the filler in terms of a number of unknown param- 
eters and then determine the parameter values that minimize the mean-squared 
error. This procedure necessarily leads to a real filter so long as the form we 
choose corresponds to such a filter. 

If we assume the special case where the input signal X(t) and noise N(t) are 
uncorrelated, (9.2-9) can be written as 


C3) 


E{e?(t)] = = | 8, (a) deo (9.3-1) 


-@ 
where 
S,tn) = Sy xl) — 2S x(w) (= ene" sb [Sy ale) + Syn) Hea)PP(9.3-2) 
Since the imaginary part of H(—w) exp (jwt,) is an odd function of w when h(t) is 
real (as assumed), the only contribution to the integral of (9.3-1) due to the 


middle term in (9.3-2) results from the real part of H(—w) exp (jet). Thus, the 
error-contributing part of (9.3-2) can be written ast 


Salto) = Sxx(@)[1 — H(@)e"/*" — H(—w)el™ + | Hw) 17) + Sant) | H(w)|? 
geal) — H(-o)el 2 + Senle) | CO)? (9.3-3) 


because H(—«@) = H*(w). 

We summarize the synthesis procedure. First, a filter form is chosen for areal 
filter. The applicable transfer function H(i) will depend on a number of unknown 
parameters. H{ew) is next substituted into (9.3-3), to obtain §,,(@), the power 


t ly writing (3-0), we also use the fact that 2 Re (2) ab 2? for any complex number 2. 
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density spectrum of the error e(t). Finally, the error E[e2(1)] is calculated from 
(9.3-1) and the parameters are then found by formally minimizing this error. 
Although this procedure is direct and conceptually simple to apply, the solution 
of the integral of (9.3-1) may be tedious. For the case where Syx(c) and Syy(@) 
are rational functions of @ and H(m) corresponds (o a real filter form, the 
resulting integral has been tabulated for a number of functions &,,(«) involving 
orders of w up to 14 (Thomas, 1969, pp. 249 and 636, and James, et al., 1947, 
p. 369). 

All the preceding discussion has related to the special case where the input 
signal And input zero-mean noise are jointly wide-scnse stationary and uncor- 
related. For the more general case of correlated signal and noise, the choice of 
form for H(m) must be substituted into (9.2-9) and the integral solved, The 
unknown filter coefficients are then determined that minimize Efe?(t)). 


PROBLEMS 


9-1 A matched filter is to be found for a signal defined by 
A(t + t)/t —t<t<0 


x(t) = (A(t — O/t O<t<t 
0 elsewhere 


4 2 
+ X(w) = At sin (91/2) 
cwr/2 
when added to noise having a power density spectrum 


as W, 
e+ w? 


Syal@) = 


where A, t, and HW, are real positive constants. 

(a) Find the matched filter's transfer function /,,,(). 

(b) Find the filter's impulse response Ngpilt). Plot Itopi(t)- 

(c) Is there a value of fo for which the filter is causal? If so, find it. 

(d) Sketch the block diagram of a network that has //,,(@) as its transfer 
function. 


9-2 Work Problem 9-1 (a), (b), and (c) for the signal 
x(t) = ute" — e847 


if > tis areal constant. 
9-3 Work Problem 9-1 (a), (b), and (c) for the signal 


x(t) = u(— fe" — 9 


ifa > tis areal constant. 


*9-4 By proper inverse Fourier transformation of (9.1-13), show that the impulse 
response /t,,,(t) of the matched filter for signals in colored noise satisfics 


| . Noo lS)R walt ze ¢) dé = x*(ty - t) 


, he eb eine DEA 
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9-5 A signal x(t) und colored noise N(t) are applied to the network of Figure 
P9-5. We select | H1,(w)|? = I/Syx(w) so that the noise N,(0) is while. We also 
make H,(w) a matched filter for the signal x,(¢) in the white noise N,(¢). Show 
' that the cascade is a matched filter for x(¢) in the noise N(¢). 


xu) +N) XU) tN (t) 
Colored Uytw) . Hy(w) xolt) +N) 
noise White 


noise 


where 1 > 0 is a real constant, is applied to a filter along with white noise with | 
power density . 19/2, Mo » O being a real constant, 
(a) Find the transfer function of the filter matched lo x(t) at time ¢,. 
(b) Find and sketch the filter's impulse response. | 
(c) Is there any value of 4, that will make the filter causal? 
(d) Find the output maximum signal-to-noise ratio. = 


9-15 Work Problem 9-14 for the signal 


etbiwe ote 


x(t) = — Del" 


Figure 29-5 


; F : ; 9-16 Work Problem 9-14 for the signal 
9-6 For the matched filler of Example 9.1-1, find and sketch the oulpul signal. 


(Hint; Fourier-transform x(t) and use a transform pair from Appendix E to x(t) = utie I 
obtain x,(t).J 9-17 Work Problem 9-14 for the signal 
9-7 Assume the power density of the white noise at the input to the matched Wi 
filter of Example 9.1-1 is Mo/2 with Wy > 0 a real constant. Find the output x() = — (= Dte 
| signal-to-noise ratio of the filter at time (,. 9-18 Ifa real signal x(t) exists only in the interval 0 <¢< T, show that the cor- | 
| ‘i 9-8 Show that the maximum output signal-to-noise ratio obtainable from a filter relation receiver of Figure P9-18 is a matched filter at time C= T; that is, show > 
| B matched to a signal x(1) in white noise with power density W 9/2 is that the ratio of peak signal power fo average noise power, both at time T, is the : 
i &. $ 2 fe 2E same as the ordinary matched filter, Assume while input noise. 
\ v 2 4 
i —_ =— | x(t) |? dt = —— \ 
| ( (i). My If No 
i tae ‘ i : xO +N) Product Teeyat Y(t) = x,t) + N,(t) 
ete where E is (he energy in x(t) and Ao > Ois a real constant. fot | : : 
_ 9-9 Lett be a positive real constunt. A pulse | 
A cos (nt/t) |e} < 2/2 AYU) 
: ' x(t) = ; : 
{0 ; |t| > 1/2 Figure P9-18 : 
is added to white noise with a power density of 9/2. Find (8,/N.)max for a filler’ é-t07%R: . : ~ 
: ee - ; the matched filler for the signal 
matched to x(t) by using the result of Problem 9-8, PEE PINE NE UNE Tee = 
9-10 Find the matched filter’s transfer function applicable to Problem 9-9. x(t) = Aewe? | 
9-11 Show that the output signal x,(‘) from a filter matched to a signal x(¢) in in while noise with power density 1%)/2 where "> 0, « > 0, and A are real 
white noise is constants. 
Fn oa 9-20 A random signal X(t) and uncorrelated while noise N(t) have autocorrela- r 
i x) = K XMQ)x(E + Ot) de tion functions 
aids 
| Fe tA tas : _ WP cana 
| That is, x,(t) is proportional to the correlation integral of x(t). Ryy(t) = ——¢ : 
9-12 Show that the output signal x,(1) from a filter matched (o a signal in while % | 
| noise reaches its maximum magnitude at ¢ = ¢, if the filter impulse response is Ryylt) = (47% 9/2)5(1) : \ 
‘ given by (9.1-15). (Mint: Use the result of Problem 9-11.) Pe en erie ee , ae 
ae 9-13 Fourier-transform the signal of Figure 9.1-ta, and use (9.1-14) (o verify the ME ee ca ares nee e ai ees Soi neree ea ye ‘ 
: optimum system transfer function given in Example 9.1-1 (a) Find the transfer function of the optimum iener fi ter, 
: P : it (b) Find and sketch the impulse response of the filter when t, < 0, ¢, > 0, and : 
| 9-14 The signal 1=0 
a . Wie dd 


x(t) = u(ge7"! 9-21 Find the minimum mean-squared error of the filler in Problem 9-20. 


a 
ge 
Ce 
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9.22 Work Problem 9-20 for colored Poise defined by 
Runt) = Wn emt 
where Wy > Ois a real constant. 


9-23 Work Problem 9-21 for the noise defined in Problem 9-22. 
9-24 A random signal X(t) and additive uncorrelated noise N(t) have respective 


power spectrums 


Sy) = oa and Syxlo) = Ga 


(a) Find the transfer function of the Wiener filter for the given signal and 


noise. 
(b) Find the minimum valuc of the error in predicting X(t + theo 


9-25 Work Problem 9-24 for signal and uncorrelated white noise defined by 
‘ 


Sax) = Ww? + «4 


Sy) = A” of 2 


where A > 0, W > 9, and A“ > Dare real constants. 

9-26 A deterministic signal x(t) = A cos (ot) and white noise with power 
density {9/2 are applied to a one-section lowpass filter with transfer function 
H(w) = W/W + jw). Here W >0, 9 > 0, Wo: and A are all real constants. 
What valuc of HW” will cause the ratio of output average signal power to average 


noise power to he maximum? 
9-27 Work Problem 9-26 if the network consists of two identic 
filters in cascade. 


9-28 Work Problem 9-26 if x(t) = 
able uniformly distributed on the interval (0, 27). 


9.29 A random signal X() having the autocorrelation function 
Ryxlt) = Wye 
AM [2 are applicd to a lowpass filler 


al one-section 


A cos (ot + ©), where @ is a random vari- 


Wylel 


and uncorrelated noise with power densily 
with transfer function 


Ha) = Gai a 


Here HW > Oand Wy > O are real constants. 


(a) What valuc of 1 will minimize the me 


be an estinate of N(t)? 
(b) Calculate the minimum mean-squared error. 


49.30 Work Problem 9-29 by finding the real constants G > 0 
filter defined by 


an-squared error if the output is to 


and W > 0 for the 


GW 


Ha) = Wo 


a 
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ADDITIONAL PROBLEMS 


9-31 A signal x(t) = u(t)St? exp (—24) is added to white noise for which A%9/2 = 
10-2 W/Hz. The sum is applied to a matched filter. 

(a) What is the filter's transfer function? 

(b) What is (5,/N.)? 

(c) Sketch the impulse response of the filler. 

(d) Is the filter realizable? 
9-32 A signal 


x(t) = ue exp (-W 
is added to noise with power spectrum 
Swnw) = PW + 0) 


where W, P, and Wy are positive constants. The sum is applied to a matched 
filter, 

(a) Find the filter's transfer function. 

(b) Find the filter’s impulse response. 

(c) What is the signal-to-noise ratio at the output? 
9-33 A pulse of amplitude A > © and duration t > 0 is x(t) = A rect (t/t). The 
pulse is added to white noise of power density W/2 when it arrives at a receiver. 
For some practical reasons the receiver (filter) is not a matched filter but is a 
simple lowpass filter with transfer function 


H(w) = W/(W + jo) 


W > Oaconstant. : 

(a) Find the ratio of instantaneous output signal power x2(t) at any time f to 
average noise power E(N2()] at the filter's output. At what time, denoted by /,, 
is the ratio maximum? 

(b) At time f, what bandwidth W will maximize signal-to-noise ratio? 

(c) Plot the loss in output signal-to-noise ratio that results, compared to a 
matched filter, for various values of 0 < W < 5/t. What is the minimum loss? 


*9.34 Reconsider the system of Problem 9-33 except assume 
H(w) = W2/(W + jo)? 


(a) Find the time f, at which output signal-to-noise ratio is largest. 
(b) For the (, found in (a) determine the oulput signal-to-noise ratio. Plot 
this result versus Wr for 0 < Wr <6 and determine what value of W gives the 


best performance. 
(ec) What minimum loss in signal-to-noise ratio occurs compared to a 


matched filter? 
9-35 A pulse 
x() = A rect (¢/20)[1 - (t/1)7] 


where A and t > O are constants, is added to white noise. 


L 


BO TEG FL 


parts Ris Ped See 


an 
i 
y 
: 


272 PROBABILITY, RANDOM VARIADLES, AND RANDOM SIGNAL PRINCIPLES 


(a) Find the output signal x,() of a filter matched to the pulse. 

(b) Sketch x(t) and x,(#). 

(c) What is the matched filter's output signal-to-noise ratio? 

(d) What is its transfer function if K in (9,1-16) is chosen so that | H,,,(0)] = 
17 Is there a value of ¢, that makes the filter causal? 


*9.36 A deterministic waveform y(t) is defined by 
W() = a(t e/Pt + oot v(t)elwot 


where a(t) and (tf) are “slowly” varying amplitude and phase “modulation” 
functions and wy > O is a large constant, The white-noise matched filter for (¢) is 
defined by 


Noplt) = W*(t, — £) 
if K = 1 in (9.1-15). Now let Y(t) be offset in frequency by an amount w, before 
being applied to the “ matched filter” so that 
Wxlt) = Y(t) exp (jw, !) 
is applied with noise to the filter. 
(a) Show that the filter's response to a(t) is 


«0 


x(t, — f, Wy) = | W(EW*(t, — t+ Eee dE 


The function | x(a, w,)|? is called the ambiguity function of the waveform W(t). 

(b) Show that the volume under the ambiguity function does not depend on 
the form of y(t) but only on | x(0, 0)[?. 

(c) Show that 


ty — f, wy) = e/wotre) [- u(é)u*(t, — t+ Eel dé 


*9.37 Reconsider the ambiguity function of Problem 9-36. 
(a) Show that | x(t, @,)|? < | x(0, 0)|?. 
(b) Show that another form for y(t, wy) is 
1 w 
x(t, wy) = a | W*(w)¥(w + we!" dw 


where ‘(w) is the Fourier transform of w(t). 
(c) Show that 


x(t, 0) = [ Wwe + 4) de 
lf wy ya jor 
-z(° | ‘P(w) Pe 7? 
x(0, w,) =|" Wg) Pent dg 
! 
~ on 


—_ ie Y*(co)'¥(w + w,) dw 
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(d) Show that the symmetry of z(t, cy) is given by 
ot, (wy) = eF"y*(—1, —w,) 
9-38 The deterministic signal 
x(t) = rect (t/T) exp (jw t + fyt?/2) 
is a pulse having a linearly varying frequency with time during the pulse’s dura- 
tion T. The nominal frequency is wg(rad/s). The matched filter for white noise has 
the impulse response of (9.1-15) which, for t, = 0, is ey 
Nop!) = K rect (t/T) exp (ja t — jput?/2) 


(a) If instantaneous frequency is to increase by a total amount Aw (rad/s) 
during the pulse’s duration 7, how is the constant j related (o Aw and T? 

(b) Find the value of K such that |H,,,(wo)| = | when yc is large. (Hint: Note 
that 


C(x) = i cos (n&7/2) dé 
0 


and 


S(x) = { sin (17/2) dé 
0 


called Fresnel integrals, approach '/, as x-» 0.) 

(c) For the K found in (b), determine the output x,(¢) of the filter, Sketch the 
envelopes of the signals x(t) and x,(0) for AwT = 80x using the same time-voltage 
axes. What observations can you make about what has happended to a(t) as it 
passes through the filter? 

*9.39 (a) Find the transfer function H,,,(w) of the matched filter of Problem 9-38, 
(Hint: Put the expression in terms of Fresnel integrals having arguments 


x, = /A@T/2n {1 — [2(w — wp)/Aw)}/./2 
x, = JAwT/2n {1 + [2(w — wo)/Aw)}/./2 


where jt = Aw/T.) 

(b) Sketch the approximate form of | H,,,(w)| that results when AwT is large. 
9-40 A random signal X() and uncorrelated while noise have respective power 
spectrums 


and 


Syy() = 2/2 Pry Wy a? Wt 4) 
and 
Syn) = ol" o/2 


Here Py» is the average power in X(t), while W, and 1, are positive constants. 
XN ge | x 0 Pp 
(a) Find the transfer function of the Wiener filter for this signal and noise. 
(b) What is the minimum mean-squared filter error? 


Bee} 
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(c) Evaluate the result of (b) for Pyy = 2 W, Wy = 15 rad/s, and of") /2 = 0.1 
W/Hz. (Hint: Use the known integral (Thomas, 1969, p. 249) 
aa (ae (by - bya?) dw 
= on |. ab + (a2 — 2agay)o? + abot Qag A,4y 
A +a, has no roots in 


gb, — 42% 
pes bea Sane 
FI 


where bo. Dy. dos is and a, are constants and ag A? + 4, 
the lower half-plane when A = @ +jo.J 
9-41 Work Problem 9-40 for the signal with the power spectrum 


Syq(eo) = AMS + wy? 
Put results in terms of the average power Pyy in X(N). 
9-42 The respective power spectrums of a random signal X 
noise N(t) arc ; 


(t) and uncorrelated 


Syylwv) = (1/20/10? + a) 


and 
Syn) = 2/162 + ay? 
(a) What is the transfer function of the Wiener filter? 
(b) What is the minimum mean-squared prediction error? (flint: Use resulls 
from Prablem 9-40.) 
*9.43 Gencralize the random signal of P 
trum is 


roblem 9-42 by assuming ils power spec- 


Syylcv) = (W74/2000)( W? + w?) 


andwidth, Find the minimum mean-squared pre- 


where My is the signal's 3-dB b 
> 9.5, What docs an inercase 10 Wy mean 


diction error and plot the result for Wy 
ina physical sense? 

9-44 A random signal X(f) plus uncorrelated noise N( 
spectrums 


1), having respective power 


Syy(ca) = 2P xg W/V E + 7) 


and 
Sylo) = 4P uy W HOV + wy 
is apptied to a Wiener filter. Here Pyy and Pry are the average signal and Noise 
powers, respectively, while Wy and Wy are positive constants. _ 
(a) Use (9.2-22) and find the filter's minimum mean-squared prediction error. 
ee ‘ 
(h) Show that as Pyy— ©, Ele) min Paw and that Ele) ]mia7? Pxx if 
Puy 0 Os 
(ec) Fron at praphical plot of Ele (Ohad Paw VERS 
there is a preferred bandwidth ratio when Pyv/P xx = 8. 
should be avoided? Discuss. (flint: Use the integral given m Problem 9-40.) 


s W/W. determine if 
Is there a ratio that 


CHAPTER 


TEN 


tee ee ee 


SOME PRACTICAL APPLICATIONS 
OF THE THEORY 


10.0 INTRODUCTION 


The main purpose of this book has been {o introduce the reader to the basic 
principles necessary to model random signals and noise. The principles were 
broad cnough to include ‘the descriptions of waveforms modified by passage 
through linear networks. In this chapter we shall apply the basic principles to a 
few practical problems that involve random signals, noise, and networks. Obvi- 
ously, the list of practical applications is almost limitless and it is necessary to 
select only a finite few. Although the applications discussed here may not heces- 
sarily serve the main interests of all readers, they do represent important applica- 
tions and do serve to illustrate the use of the book's theory. 

In the following scctions we shall describe two practical communication 
systems, two control systems (one with application to one of the communication 
systems), an application involving a compuler-type signal, and two applications 
that relate to radar. In every case we are primarily interested in how these appli- 
cations are affected by the presence of random noise. We begin by considering 
the common broadcast AM (amplitude modulation) communication system. 


10.1 NOISE IN AN AMPLITUDE MODULATION 
COMMUNICATION SYSTEM 


The communication system most familiar to the general public is probably the 
AM (amplitude modulation) system. In this system the amplitude of a high- 
frequency “carrier” is made to vary (be modulated) as a linear function of (he 
message waveform, usually derived from music, speech, or other audio source, 
The carrier frequency assigned to a broadcast station in the United States is one 
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of the values from 540 to 1600 kHz in 10-kHz steps. Each station must contain 
its radiated power to a 10-kHz band centered on its assigned frequency. 

In this section we shall give a very brief introduction to the AM broadcast 
system and illustrate how the noise principles of the preceding chapters can be 
used to unulyze the system's performance. 


+agly 


5a) 


AM System and Waveforms 


Figure 10.1-1 illustrates the basic functions that must be present in an AM 
system. In this figure we include only those functions necessary to the study of 
noise performance. A practical system would include many other devices such as 
amplifiers, mixers, oscillators, and antennas that do not directly affect our per- 
formance calculations. 
The transmitted AM signal has the form 
Sam(t) = [Ay + x(0)] cos [wot + Oo] (10.1-1) 
where Ay > 0, wy, and Oy are constants, while x(t) represents a message that we 
model as a sample function of a random process X(t). Note that the amplitude 
[Aq + x(t)] of the carrier cos (wot + Uo) is a linear function of x(t), Now, in 
general, one has no control over Vy because the turn-on time of a transmitter is 
random and the channel itself may introduce a phase angle that is random 
(which we presume is absorbed in the value of 09). Thus, we may properly model 
Mg as a value of a random variable © independent of X(t) and uniformly distrib- 
uled on (0, 27). These considerations allow say(t) to be modeled as a sample func- 
tion of a transmitted random process S4,,(t) given by 
Saml) = [Ao + X(t)] cos (wot + Oo) (10.1-2) 
The transmitted signal arrives at the receiver after passing through a channel 
with gain G,,. The channel is assumed to add no signal distortion but does add 
zero-mean white gaussian noise of power density “9/2. A practical channel typi- 
cally adds delay but this effect does not modify the noise performance. A receiver 


bandpass filter passes the received signal sp(t) = Go, Saq(t) with negligible distor- 
tion but has no wider bandwidth than necessary so as to not pass excessive 


Salt) + aft) 


RECEIVER 


Channel voltage 
gain= Ga 


White guassian noise. 
power density = MR 
+ | 
Saul Lees 
+ 


noise,t The noise a(t) at the filter's output is a bandpass noise so the theory of r SSS SSS STs 
: Section 8.6 applies. ei = 
2 We model waveforms s,(t) and n(t) as sample functions of processes S,(1) and E 5 + 
lee N(t), respectively. Thus, we may write | z 3 ? 
is A 2 | 
aan SO = Goa Sau) a 3 8 | 
4 ee 
re = Gallo + X(t] cos (wot + Oo) (10.1-3) 
I N(t) = N,(t) cos (gt + Oo) ~ N,(t) sin (9 f + Oo) (10.1-4) tl. -----~--4----~-— == “ 
rn where N,(t) and N,(t) are lowpass noises with average powers N2()) = Ni() = : 


red 


NI) from Section 8.6. 


Figure 10.1-1 Functional block diagram of a broadcast AM system. 
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t The required bandwidth ¥,,, must be at least twice the spectral extent HY, of X(0). 


277 


Se Bae a ate eae eS es 


278 PRODARILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES 


Noise Performance 

A good measure of noise performance is the ratio of the average power in the 

output signal s4(4) of the system to the average power in the oulput notse nt). In 

the AM receiver an envelope detector is used to recover the transmitted message. 
The total waveform applied to the envelope detector becomes 


S(t) + N(O = (GenlA0 + X(N] + NO} cos (Mo! + Oo) — Nyt) sin (ot + ©») 
= A(t) cos [wot + Oo + W(t] (10.1-5) 


wherct 


N (0) 
ciate holt NUD aoe (10.1-6) 
w(t) = tan loa + X(Q) + al . 


Att) = ((GalAo + XM) + NO}? + NO)? 


2NAt) N20) + N20 ‘i eae 
bad Gal Ao ca ven( + Gal Ae j X(N] + GL Ag y: xO? ( . ) 


Now only (10.1-7) is of interest because A(!) is the envelope of S,(1) + NU). The 


detector outpul is this envelope. 
Since N2(t) + N21) is the instantaneous envclope of the square of N(t) 


(related to received noise power), while G2,[A0 + X()}? is the instantancous 
envelope of the detector's iaput signal (related to received signal power), we make 
the assumption that input (received) signal-to-noise power ratio is large so that 
{N2(t) + N20)1/G30A0 + X(d]? is small most of the time. The assumption allows 


A(t) © Galo + X()1 + Ne) (10.1-8) 


from (10.1-7) Only when this condition is truc do we obtain quality performance 
anyway, so other situations are not usually of interest. 
If we model s(t) and nt) in Figure 10.1-1 as sample functions of processes 
St) and NAO, respectively, then (10,1-8) clearly gives 
St) = Geno + X10] (10.1-9) 
NA) = NC) (10.1-10) 


The useful output signal average powcr, denoted by S,, is that duc to X(A) in 
(101-9). output average noise power is denoted by N, then 


s, = GX (0.t-H) 
N, = Not) = NACI) (10.1-12) 


{ Vypically, overmadulation where LXE uae the maximun magnitude of Xu) exceeds Ay 18 ande- 


sirable in AM, so [4a | N(t)] > (is assumed in (101-7), 
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and performance is measured by 


S, G3, X7(!) 
=) =«-h— 10.1-13 
(),, N21) ( ) 


Next, we model the bandpass filler in Figure 10.1-1 as an ideal filter with 


bandwidth W,,, (rad/s). Noise power readily follows 


Base. il 0 + (Weeel 2) NeW 
N B= 12] (A’ 9/2) do = —@—# (10.1-14) 
20 Jorg —(Weeel2) 2n 


From (10.1-13) we have 


12 ayy 
(=) ota (10.1-15) 


NoJam 0 Meee 


Lquation (101-15) is the principal result of this section. It describes the per- 
formance of the AM system. It is helpful to demonstrate the use of (10.1-15) by 
means of an example. 


Example 10.!-1 Assume an AM system uses an unmodulated carrier of peak 
amplitude Ap = 10./95 V and a message of power X(t) = 500 W. Its 
channel has a gain Ge, = /32/100 with a noise density M 9/2 = (107°) 
W/liz. The receiver uses a filter with bandwidth W,. = 2n(10*) rad/s. We 
compute various signal powers and system performance. 

From Problem 10-1 the average power in the transmitted carricr is 
Az/2 = 4750_Wy the transmitted power due to message modulation is 
Ryx(0)/2 = X()/2 = 250 W. Total average transmitted power is, therefore, 
5000 W. , 

From (10.1-15) we compute 


S, 2n(32)10~*(500) 
(=). = 10~8)2n(10°) = 8000 (or 39.03 dB) 
This signal-to-noise ratio represents fairly good performance. 

At the input to the envelope detector the reccived average signal power is 
5000 W_ decreased by the loss incurred in passing over the channel: 
5000( /32/100)? = 16 W. From (10.1-14) and (10.1-12) the inpul average 
noise power is 10782n(104)/a = 2(107*) W. Input signal-(o-noise ‘ratio 
becomes 16/2(107*) = 80,000 (or 49.03 dB). This valuc is well above the 
ininimum for performance as required for (10.1-15) to be valid: in fact, if the 
performance of an AM system is satisfactory then (10.1-15) will always be 
valid (the reader should justify this fact by examining the efficiency of an AM 
system -~sec Problems 10-4 and 10-2.) 
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10.2 NOISE IN A FREQUENCY. MODULATION 
COMMUNICATION SYSTEM 


Another communication system with which the reader is familiar is the broadcast 
FM (frequency modulation) system, Here the instantaneous frequency of a sinu- 
soidal “carrier” waveform is made,to vary as a linear function of the message 
waveform. If X(t) is a process repfesenting the message, the FM transmilted 
waveform can be represented by the Focess : 


uP. 
Sen(t) = A cos [oot Oo + ken [xo a] (10.2-1) 


where A, Wo, and ky, > 0 are constantst and ©, is a random variable indepen- 
dent of X(t) and uniformly distributed on (0, 22). In a practical station co4/27 is 
the station's assigned frequency and is one of 100 possible frequencies from 88.1 
to 107.9 MHz, Each station transrhits power in a 200-kHz “channel” centered 
on ils assigned frequency. , 

The constant ky in (10.2-1) is the transmitter's modulation constant. Its unit 
is rad/second per volt when X(t) is a voltage. Transmitted signal bandwidth is 
difficult to compute in FM because FM is a nonlinear modulation. If ke, is large 
enough, this bandwidth can readily be much larger than the bandwidth of the 
message process X(t). If X(t) is presumed to be bounded at | X(t)|ma, and have a 
crest-factor defined by (Problem 10-3) 

IX) nex — LX) Vina 


2 phe aA L113 
Ku = 


ELX*()) x %(1) 


the bandwidth of Syy(t) for the broadband case is approximated by (Pccbles, 
1976) 


(10.2-2) 


Wen & 2h = keg] X (2) lnax 
= 2keu Ken X(t) (10.2-3) 
Here 
Aw = key | X(t) bn (10.2-4) 


is the peak frequency deviation that instantaneous frequency can make from apy 
(on either side). 
Although dificult to prove, the average transmitted waveform power is 


2 


A 
Prey = ECS2a(O)] = a (10.2-5) 


which is independent of the modulation. 


t Ik, is negative its sign can be absorbed into the definition of X(0). 
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KM System and Wayeforms 


Figure 10.2-1 illustrates the basic functions present in a typical FM system. The 
transmitted waveform passes over the channel modeled as a power gain G3, 
without distortion or delay (as also assumed in Section 10.1 above), The receiver's 
bandpass filter (BPF) is wide cnough to pass Gy, Seal) with little distortion but 
not so wide as lo pass excess noise, Its bandwidth is, therefore, Mey = 2Aw. 

The purpose of the limiter is to remove amplitude Auctuations in the 
received waveform. The limiter is necessary so that the receiver responds only to 
frequency variations (that contain the message) and not to amplitude variations 
that are mainly due to noise. The discriminator is the actual demodulation 
device; it produces a voltage proportional (constant of proportionality Ky) to 
instantaneous deviations of the frequency of its input waveform from a nominal 
value wa. Ideally, with no noise, the diseriminator’s output signal is Ky kyy X (0. 
The lowpass filter must pass this waveform with low distortion so that its output 
is proportional to X(1) 


SQ = Ky Keay X(0) (10.2-6) 


It should have a bandwidth no wider than the spectral extent of X(N, denoted by 
Wy, So as to nol allow excessive output noise. 

If the receiver's “input” is defined as the input to the limiter, the input 
signal's average power S; is 


Ae 
S; = G3, ae (10.2-7) 
2 
while the output signal power is 
S, = E{S3(1)) = Kb kim X70) (10,2-8) 


By modeling the BPF in Figure 10.2-1 as an ideal filter the input noisc power 
is readily found to be 


1 wo td Ac {7 A 
N,=—2 { Sige (10.2-9) 
2n Wwo 7 dee n 
Input signal-lo-noise power ratio is 
. 1242 
De Gees, (10.2-10) 
NiJin 29 Aw 


from (10,2-7) and (10.2-9). 

Computation of output noise power is less straightforward than the preced- 
ing computations, However, its development forms the most interesting problem 
in computing system performance. 


og 
ore 
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FM System Performance 


For large input signal-to-noise ratio we have |G.,A| > IN.(O} and |G, A] > 
LNAO| most of the time, so (10.2-13) becomes 


Sin NAO 


¢ = Care must be exercised in finding outpul noise power because FM is a nontinear 
Hee operation. For relatively large (S/Nidim and wideband aperation (developed 
és" abeve), signal and noise powers may be independently found, Signal power is 
\t aneeen found assuming noise zero (above). Noise power is found assuming the message ; 
[2 [Se 1 is zero but carrier is still transmitted. In this latter case the waveform at the ay 
| % limiter is i 
| a. % 
‘i £3 ; & 
4: | | 52 Gu, A 008 [0 ! + Oo] + Nell) COS (Wot + Oo) — N,(t) sin (wot + Oo) 4 
se 
| = A(t) cos [wot + Oo + H(] (102-11) “a 
Wy | ° 
je | where the bandpass noise Nt) is modeled as in (10.1-4) (see also Section 8.6) and i 
3 | 
A(t) = {{Gen A NAO]? NZD}? (10.2-12) 
re I N,(0) 
‘ | t)=tan7! {5 10.2-13 
i oe HO = Gn VGA + NO 
| 
| 
| 
| 
| 
| 
\ 
| 
| 
| 
ie 


' a 3 : 
| i 1) NCO N,(t) 
)xtan 7! | [a 10.2-14 
2 i la Al Ge, A ( oe ) 
8 
2. Equation (10.2-11) is now approximated by 
c 
¢ : 2 Nt) 
j f eo 2 A(t) cos [wo f + Oo + W(t)) & Ald) cos | Wot + Oo + GA (10,2-15) 
Th a 3 ch 
. a” 5 E 
" : a x ; Because the limiter removes A(t) and the discriminator responds only to instanta- 
y' ae 2 > neous frequency deviations from Wo, the input to the lowpass filter is 
My z 3 3 ae 
: b g%& § 
‘ ¢ as IN 
: fk ge 3 (f ) os (10.2-16) 
ve } ¥ § Cun A dt 
ch i Sh 
mo 3 If Sy,w,(@) is the power spectrum of N,(t) the power spectrum of (10.2-16) is 
av} 
i 3 Kp 5 2 
i = wSy,n(w) (10.2-17) 
; » € Gen A 
F - 3 2 
{i 3 5 However, we may use (8.6-17) and (8.6-16) to write this power spectrum as 
i E ~ 2 
in 3 K 
= (2 2) co*[Syiw(@ — @o) + Syl + Wo)] — |w] < dw (10,2-18) 
{; a 3, 7 he 
t = te where Sy,v(m) is the power spectrum of Nr); iC is constant at 47/2 over bands ( 


of width 2Aw centered at wg and —wo. 


i 2H2 


£ 
e: 
e 
Be 
x 
fe. 
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Final output noise power results from the action of the lowpass filter on 
(10.2-18). We have 


] Wry K 2 
N, = ELN2()] = a [ ( B ) (Syn (@ — Wo) + Syn {@ + W9)] deo 


Wey GyA 
K} Wr Mo , Mo KiW, Wi 
“FGa [2 be a aaGh Aa ee 
Output performance is determined by 
2 2p2 2 
So\  _ 3nGen Athi X*() (10.2-20) 
N, FM No Wi 


from (10.2-8) and (10.2-19). An alternative form of (10.2-20) is 


S; 6 {Aw\3/S,\ 
Po =—~-|—)]{—t 10.2-21 
(),. Ke, (i) Ga ( 


An important observation derives from (10.2-21). Since FM bandwidth is 
2Aw, we see thal performance increases as the cube of bandwidth relative to 
(S/Nodum-~ However, (S/N ajay decreases as the reciprocal of bandwidth from 
(10.2-10), so the net performance increases as the square of bandwidth. By simply 
increasing bandwidth at the transmitter, system performance rapidly increases. 
There is a limit to this procedure, unfortunately, that occurs when conditions 
under which the performance equations were derived are no longer valid. The 
break point, or threshold, occurs approximately where (S/N)iaq drops below 
about 10 (or 10 dB). For a more detailed discussion of FM threshold the reader 
is referred to Peebles (1976). We shall emphasize FM system performance 
(through an example, 


Example 10.2-1 An FM system uses a message with crest factor 3 and band- 
width Wy/2n = 3 kHz The FM modulator’s bandwidth is 2Ac/2n = 20 kHz, 
and the receiver's input signal-to-noise ratio is &8t. From (10,2-21) 
(SU/N gene = 2000 (or 33.01 dB) We determine how much performance can be 
increased by raising Ac. 

From (10.2-10) (S/N dpa, deercases Co FO frony 82 if Aco inercases by a 
factor of 8.1, Next, we again use (10.2-21) but now with Aw/2n = 8.1(10) kHz 


and (S//NjJru = 10: 
Ry 6 /81\? 
pet 5 = —-(— 10) = 
GE 9 (5) ( °t ee 


(or 51.18 dB). The bandwidth increase of 8.1 times has improved (S.,/N,)yx 
by 65.61 times. 
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10.3 NOISE IN A SIMPLE CONTROL SYSTEM 


In this section we shall briefly consider the noise response of a simple control 
system modeled by the block diagram shown in Figure 10.3-1. The following 
section will then illustrate how a very practical network can be analyzed by 
applying the results developed here. 


Transfer Function - 


Typical loop behavior in Figure 10.3-1 is to force the feedback signal F to 
‘approximate the command C so that the error C-F is small, The control loop’s 
response R may be conveniently chosen. For example, if R in the time domain is 
lo be the derivative of the command then H,(w) = I/jw, the transfer function of 
an integrator. If R is to approximate C then F,(w) = |. 

From Figure 10.3-1 it is clear that 


R(w) = Hy (w)( Cw) — Hy (w)R(w)] (10.3+1) 


so 


tt | (10.3-2) 


R(w) = cial + H,(w)H (w) 


We define the transfer function of the control loop as 


_ Rw) _ H,(w) : 
Ho) = "TH taHilel (10.3-3) 


The transfer function (10.3-3) is not always stable. There are combinations of 
H,(w) and H,(w) that can cause instability, In general, if H,(w) and H,(@) are 
stable and | H1/,()/1,(w) | falls below unity, as a function of w, before the phase of 
1 ,(@)H,(@) becomes —2, and if the phase of H,(@)H (cw) equals ~— 7 at only one 
frequency, the transfer function H(w) is stable. The product H ,(w)I,(w) is called 
the open-loop transfer function of the control system, Stability is a deep subject in 
control systems and we shall not develop it farther because it detracts from: the 
simple points (o be made here. 

Now suppose the command waveform in Figure 10.3-1 is the sum of a signal 
3.0) and noise V,(0. Because (he system is linear its responses (o signal and noise 
may be computed separately, If Sy,,(w) is the power spectrum of N,(t) then the 
power spectrum of the response noise N x(t) is 


H ,(@) 


T+ Ho) (@) (10.3-4) 


Swynyl®) = Syne) 


whenever the network is stable. 
An example serves to illustrate the use of (10.3-4). 


vee 
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F(w) 


Feedback 
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Figure 10.3-1 Block diagram of a simple control system. 
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oe ee 
Example 10.3-1 Let a signal 
S{0 = Au(te 


plus white noise of power density A” 9/2 be applied to the contro! network 


where 
K,W, 


H (wo) = = 
1(cv) W, + jin 


K,>1 


Iw) = 1 


This choice means that we desire the response to equal the command, We 
find the output signal and the ouput noise power. 
From (10.3-3) 
H(w) = K WM tio) KW, 
1+ (K,WAW, + joy) (b+ KM + jo 
From pair 15 of Appendix E the inverse transform of H(«) is 
h(t) = Ky Wyu(ge (teow 


The response signal becomes 


Sxl) = | H(A (t — &) do 


= K,W,A { u(Zu(t — eye 7H KOM IS dee 


=m 


= KW, Auge ee a fa 
0 
KW, 
rere ETT ERAT ie —{(l + KY, - WI): 
Tecra Ee MW, = VIDS 
For K, > Uso that (1+ Ky) > W this resull becomes 
Salt) = S.C) 


The approximation is more accurate as ¢ becomes large. 
From (10.3-4) the output noise power spectrum is 


_ Ak 
Swann) es ce! + K,)W,)? + w? 


Output noise power is found using (C-25): 
oe ous 
Puana = on [ _Bwaanl) da 


_ AQ KIM, tg Ki MH 
“41+K,) 4 
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We observe in passing that this control loop is stable and its transfer 

. function is equivalent to a simple lowpass filler of gain Ky/( + A) & 1 and 

3-dB bandwidth (1 + K,)W, » K,W,. This unity-gain large-bandwidth filter 

resulted from a narrowband (bandwidth W) high gain filler (gain K,) inside 
the loop. 


Reaiire Shep heed cee acre 
=a = z 


DP renepycceppraksa eer ee 


Error Function 


Output signal 


= 
The error Q = C — Fin Figure 10.3-1 is readily found. From 
. Q(w) = C(w) — F(w) = C(w) — H(w)H ,(@)Q(w) (10.3-5) 
we have ; 
nee) 6) 
QU) = Tr coy lea) (103-6) 


Wiener Filter Application 


By comparing (10.3-3) with the transfer function of a Wiener filter for uncor- 


related signal and noise as given by (9.2-20) we see that the Wiener filter can be 
implemented as a loop. From (9.2-20) 


elite 
H,,(@) = ———_—__———- 10,3-7 
me) TF Coun Sua] aes 
Thus : 

H(cw) = H yy) (10,3-8) 
ye if = 
Wot + 
Boa; H(w) = el (10.3-9) of 
x be 
ee H (2) = [Syy(w)/8xx(w)]e- 1°" (10.3-10) Ae 
z Of course (these functions H,(w) and H,(w) may not be realizable even for realiz- ; § 
able signal and noise power spectrums. Other choices for H,(w) and H (cw) arc aX 


also possible (Problem 10-10). 


10.4 NOISE IN A PHASE-LOCKED LOOP 


Pe 
A, OS [ergt + 99 + Oft)) 
= A, cos (8, (2)} 


Input signal 


ae 
a 


The phase-locked loop (PLL) is a practical system to which the noise theory of 
this book can be applied as a good example. The PLL is also an example of the 
control system of the preceding section. 

Figure 10.4-1 depicts the block diagram of a PLL. Broadly, the action of the 
loop is to force the phase of the output of the voltage-controlled oscillator (VCO) 


PUREST 


Figure 10.4-1 Block diagram of a phase-locked loop (PLL). 
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to closcly follow the phase of the input signal. This action leads to onc of the 
most impertant uses of the PLL, ‘that of demodulating a frequency-modulated . 
signal. If there is no input noise N{t) and the VCOs phase follows that of the 
input FM signal, then the VCOs signal has the same FM as that transmitted. 
Since the VCO is just a frequency modulator, its input waveform (loop’s outpul 
waveform) has to be proportional to the original message used at the transmitter. 
When input noise is present there is noise on the output signal. In this section we 
shall develop this output noise power and find the available output signal-lo- 


noise power ratio. 


Phase Detector 


Consider first the phase detector. Although there are many forms of phase detec- 
tor [Blanchard (1976) and Klapper ct al. (1972)] they all provide an oulpul 
response proportional to the difference between the phases of the (wo input 
waveforms for small difference phases. Thus 


en) & Kel) — 0209] (10.4-1) 


if the wo input waveform’s phases are defined as 0,(t) and 0,(0). The constant Kp 
is the phase detector's sensitivity constant, its unit is volts per radian for e,(t) a 
voltage. In some phase detectors the response is also proportional to the ampli- 
tudes of the two input waveforms. Others depend only on one input amplitude 
beenuse the other is large cnough to saturate the device giving a type of limiting. 
Another type allows both inputs to limit in the detector and the outpul is nota 


function of cither input waveform’s level. We shall assume either this last form of 
gnal’s input when a detec- 


detector or that an actual limiter is in the path of the si 
tor is used with limiting in the feedback path’s input. Thus our phase detector is 


described by (10.4-1). 


Loop ‘Transfer Function 


since the VCO in: Figure 10-4-1 acts like a frequency modulator for the 
“message” salt), ils output can be written as 


VCO output = Ay Cos [wot + 09 + Oty) 


Ay cos [oot +09 + ky { sx a| 


= A, cos [0,(0)) (10.4-2) 


i] 


where ky. is the VCO's modulation constant, 


0,(0) = wot + Oo + ky {so dt (10.4-3) 
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and 
OY(t) = ky [ suo dt (10.4-4) 


The other phase detector input signal, from Figure 10.4-1, is the input wave- 
form. If we define its phase as 


O,(t) = cog! + Op + O41) (10.4-5) 


then the phase detector’s response (10.4-1) becomes 
e(t) = Kf ot + 09 + Ot) — Wot — Oy = ky [sx a] 


= Ke 040 sche [ san av| (10.4-6) 


Next, if we define Fourier transforms as follows 


e,(t) + E,(w) (10.4-7) 
0,1) + O{e) (10.4-8) 
Salt) > Salo) (10.4-9) 

we may write (10.4-6) as 

kyé 
E,(w) = K for - ‘Sat (10.4-10) 
jw 
From Figure 10.4-1 
i Sp(w) 

E, (a) = TI, {) (10.4-11) 


On cquating (10.4-10) and (10.4-11) we find the PLL's transfer function, denoted 
by H,{«), to be 


Sa) Kpjmlt fo) je 


Hw) =o = ‘ 
7() Ow) jo + Kp ky H 1 (ev) ky 


H(w) (10.4-12) 


where we also definet 


Kpky H,(w) 


H(w) = —Eot 
ja + Kpky H{o) 


(10.4-13) 


t In many texts H(c) is called the PLL transfer function but the toap’s output is defined at a dil- 
ferent point. (Where would it be?) 


‘ 


Phase modukedion 
of input waveform \ 


a(1) 


Ow) 
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Kyl, (w) set) 


Naylor) 


Phase modulation 
af feedback waveform 


8,-(1) Voltage-controlied 
Oy (w) oscillator 
(a) 
iis Phase-locked loop 
AU 
Hy(w) = jall(wlky he 
. w 
),(w) Pe # 


(h) 


Figure 10.4-2 (a) Equivalent block diagram of the linear PLL of Figure 10.4-1, and (h) the transfer 
function equivalent of the loop in (a), 


lt should be noted that the above definition of (ransfer function relates the 
output signal to the input signal’s phase modulation 0{t) according to 


Sa(w) = H7(w)O(w) (10.4-14) 
or 


Salt) = { Naft ~ O)0(¢) de (10.4-15) 
where A(t) denotes the inverse transform of H,(w) 


hy(t) + Hw) - (10.4-16) 


The above developments show, in effect, that Figure 10.4-2 is an equivalent 
form for the loop of Figure 10.4-1. 


Loop Noise Performance 


We shall apply the preceding results to the case where the input to the PLL is the 
sum of an FM signal plus bandpass noise N,(t) modeled as 


Nit) = N,(t) cos (wot) — N,{t) sin (wo!) (10.4-17) 


The representation (10.4-17) follows developments of Section 8.6 where N,(t) and 
N,(t) are lowpass random processes having the properties defined in (8.6-7) 
through (8.6-19). The actual input to the PLL is, therefore, 


A, cos [ost + 00 + Keay | x0 a| + N.{t) cos (Wot) — N,(t) sin (wot) (10.4-18) 


Output waveloun 
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where kyq is (he FM modulator'’s coustint, V(t) is the messiaye process, and <f, 
Wg, and 0g are the input FM signal's peak amplitude, frequency, and pliase, 
respectively. 


The exact analysis of the PLL’s response to the waveform of (10.4-18) is very 
involved. However, it can be shown that the waveform of (10.4-18) can be put in 
the form (Problem 10-11) 


R(t) cos [wot + Oy + Oral) + On(t)] (10.4-19) 


where 
Drerglt) = Kens | X(t) dt (10.4-20) 


and 0,(0) is a phase angle caused by noise. Mor large-input signal-to-noise rade 
(A2/2VELN2()) and input noise N{t) broadband relative to the FM signal, the 
autocorrelation function of 0,(0 is approximately 1/A? times the autocorrelation 
function of N,(t) (Problem 10-12), This fact means that, within a reasonable 
approximation, 0,(t) can be replaced by the equivalent angle N.(0/A;. 

With the above noise equivalence used, the input phase modulation to the 
PLL from (10.4-19) is 


Ot) = Oyaglt) + On(t) 


Nt 
= Os-ny(t) + Net) (10.4-21) 
A 
i 
The component 0,,,(f) is due to the signal. If X(t) is a random process with power 
spectrum Sy(w), we use (10.4-20) in (10.4-21) and find that the power spectrum of 


Ot) is 


ky 8 Sun d@ 
Sojo(co) = EERE! vate aw Cas (o) (10.4-22) 
i 


After using the PLL's transfer function (10.4-12), the output waveforim’s power 
spectrum becomes 


Syern(@) = Soo(@)| H() |? 


k y 2 2 
= 5,a(o(2) | H(w) |? + Sy, (0) a | H(w)|? — (10,4-23) 
ky Apky 


The first right-side term in (10.4-23) is due to the desired message while the 
second is duc to noise. Loop design is typically chosen so that | H(w) |? = 1 for all 
frequencies of interest in Syy(w). In fact, if the message is to be preserved with 
very small distortion the bandwidth of the transfer function H(cw) may be signifi- 
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an the frequencics of interest in Syx(). Thus, if W, is the spectral 


cantly larger th 
he output signal component is 


extent of the message X(/) then the power in t 


S Pas Sy x(a) Kea “Hwy? dw kw? |” (co) den 
ea Qn et xN ky ky Qn ie xXx 


. \2 
= (%) KUO (10.4-24) 
: : 
to follow) | H(w)|? does not decrease rapidly cnough 
oise duc to the factor wt in wf (a)? in (10.4-23), In 
lo follow the loop with a separate filter to better 
W,. As long as cither the 
he overall output noise 


In some loops (sce example 
to remove high-frequency 0 
these cases it may be necessary 
remove noise spectral components at frequencies |«| > 
loop or a separate filter removes these components, t 


power is approximatcly 


{ Ny 2 
Nowe | Syn (eo) pce Lica)? den 
“ an NeNe y 


x Pk | 

Wy ( A, WR 
sepsis res seule 0.4-28 
nA? ke [i OO Sa At ke Meee) 


r ratio from (10.4-25) and 


Finally, we determine oulput signal-(o-noise powe 
plitude of the transmitted 


(10.4-24). As in Section 10.2, we let A be the peak am 
FM signal and let Ger be the gain of the channel, so that 


A, = AG (10.4-26} 
Thus, 
‘ 3 7 . 3 f 2 
Sa), 38Gin APhiw NU) (10.4-27) 
Nu/vM Mo Wy 


10.2-20) we find that both the discriminator and 


On comparing (10.4-27) with ( 
the same performance when the received (input) 


PLL forms of FM receiver have 
signal-(a-noise ratio is large. 


Example 10.4-1 As an example of a practical PLL's transfer function let the 
loop filter be a simple low-pass function with 3-dB bandwidth HW, where 


W, 


= — 
fag) W + jo 


The function [t), from (10.4-13), becomes 


{ 


Ia) = —s NT TT NN 
SOnEe 
O, a, 
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Olog Hfw)!, dB 


+ 


ht 0.2 ad 0 24 4.0 


wl, 


Figure 10.4-3 | (| for the loop of Example 10.4-1. 


where the quantitics defined by 
w, = (Kpky w,)'”? 


1 |W, 
a Kpky 


are called the natural frequency and damping factor, respectively, of the loop. 
Figure 10.4-3 illustrates how | H(w)| behaves with w/w, for C as a parameter, 
The curve for ( = 1/./2 is most Nat in the sense that the largest number of 
derivatives of | H(w)| are zero at w = 0.,. 

For(= 1/./2 and w, = Wy, the signal's spectral extent, we have 


4 


Ww 
H(w a 2 
| H(a)| Wit o 


The more exact power in the noise term of (10.4-23) becomes 


_MoWx [° w? dw wl We 
a marrey iT Te cae TOT 
2nA2k2 |. Wy tot 2/2 ATK R 


after using (C-38). On com aring this result with (10.4-25) we see the noise in 
the loop output is 32/2./2 = 3.33 times that of a broadband loop follawed 
by an abrupl-cutoff filler of bandwidth Wy. 


iene eee eetoee 


ss 
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10.5 CHARACTERISTICS OF 
RANDOM COMPUTER-TYPE WAVEFORM 


As another example of the practical application of the theory of this book we 
examine a waveform not unlike those encountered in binary computers. The 
waveform is shown in Figure 10.5-1; it consists of a sequence of rectangular 
pulses of durations T, having amplitudes that randomly may equal A or —A. 
Amplitudes A and —A are assumed to occur with equal probability and the 
amplitude of any pulse interval is assumed to be statistically independent of the 
amplitudes of all other intervals. The random process from which this type of 
waveform is modeled as a sample function is called a semirandom binary process 
(sec also Problem 6-4); in the remainder of this section we shall examine the 
description, power spectrum, and autocorrelation function of this process, 


Process Description 


The semirandom binary process X(‘) can be described by 


Xi)= So Ay rect [=] (10.5-1) 


heiow b 


where {A,} is a set of statistically independent random variables and rect (-) is 
defined by (E-2). The A, satisfy 


E{Ax] = 90 k=0, +1, £2,... (10.5-2) 


At ok=m 


-3 
0 kAm C0529) 


ELA, Ay] = 


The truncated version of X(t) is needed in calculating power spectrum. We 
truncate to a time interval 2T centered on ¢ = 0 that is a discrete multiple of 7, 
according to 


21 = (2K + TG (10,5-4) 


Figure 10.S-1 Typical waveform of a semirandom binary random process. 


SOME PRACTICAL APPLICATIONS OF THE THEORY 297 


Thus, the truncated process X (0) is 


x opp 
VA0= ¥ Ay rect [=] (10,5-5) 


ka-k b 


Power Spectrum 


We compute the power spectrum Syx(w) of X(1) by use of (74-11) The Fourier 
transform of X (1), denoted by X,(@), is 


Kx 


Xw) = T YL Ay Sa(wT/2)e78" 
ka~k 
x : 
= 7,Sai'l,/2) YoAge Mele (10.5-6) 
ko-K 


from (10.5-5) and pair 5 of Table E-1. Next, 


: feo) 2 Sata 2) & K - 

EEX to) Suey Tf ) s > ELA, A, Jer Mee mee 
2T (2K 40) pltg mete 
= A*T, Sa*(wT,/2) (10.5-7) 
Now because (10.5-7) does not depend on K, and therefore not on T through 
(10.5-4), we have 
«ELL Xs(e) 7) 
$yx(@) = lim ——— 
XX Sica 2 T 


The bandwidth of this power spectrum at ils —3-dB point is ().4429(2n/1,) = 
0.44290y,. 


= AT, Sa*(wT;/2) (10.5-8) 


Autocorrelation Function 


It follows from (10,5-1) through (10.5-3) that ELY()X(¢ + 1)] is zero unless both ¢ 
and ¢-f t fall in the same pulse interval. The autocorrelation function is, there- 
fore, 
Ryxl, Ot) = EEY(ON (CO) 
ie (k — YA)l, <(tand t+ 1) <(k + ANT, 


(10.5-9) 
() elsewhere 


Thus, the process X(t) is not even wide-sense stationary since (10.5-9) depends on 
absolute time f. 

The time-averaged autocorrelation function is readily obtained by inverse 
Fourier transforming (10.5-8) according to (7.2-9). After using pair 7 of Table E-1 
we oblain 


ss { 
Ryy(t) = lim = | Ryy(t, C+ 0) dt = AP tri (=) (10.5-10) 
, 
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The direct computation of R yx(t) by time-averaging Ryx(t, 6 + 0) is possible, but 
a bit: more complicated than the inverse transform procedure used here (sec 


Thomas, 1969, p. (07). 


10.6 ENVELOPE AND PHASE OF A 
SINUSOIDAL SIGNAL PLUS NOISE 


Many practical problems involve the probability density function of the envelope 
of the sum of a sinusoidal signal and noise. A radar, for example, may be inter- 
ested in determining if a short segment (pulsc) of a sinusoidal waveform is being 
received at some time or if only noise is being received. This problem is once of 
detection based on observing the received waveform's envelope; if the envelope is 
large enough (because of the signal's presence) the radar decides both the signal 
and the noise are present. We examine radar detection further in Section 10.7. 

In this section we discuss probability densitics involved in describing the 
envelope and phase of the sum of the sinusoidal signal and noise. 


Waveforms 
Let the signal be 
s(t) = Ag COS (ol + Ug) = Ag COS (Wg) COS (ag 0) — Ao sin (0g) sin (2) 0) (10.6-1) 


where Ag. Mp. and Oy are constants. We assume the noise n(t) to be added to s(t) 
is a sample function of a zero-mean, wide-scnse stationary gaussian bandpass 
process N(!) with power ECNA()) = o?. From (8.6-2), the sum can be writen as 
s(t) + N() = (Ao Cos (Oo) + X(t)} cos (9 t) — [Ag sin (Qo) + Y(A)] sin (4 
= Rit) cos [Wot + O(!)) (10.6-2) 


where X(t) and (0 are zero-mean, gaussian, lowpass processcs having the same 
powers ECX7(1)] = ELY2(0)) = ELN7(1)) = 2, Other propertics of X(/) and Y(t) 
are given in (8.6-7) through (8.6-19). The envelope and phase of the sum are R(t) 
and ©(1), respectively. We may think of R(t) and O(4) as transformations of X(t) 


and ¥( as follows: 
“R= TUN, ¥) = (Ao 608 (Oo) + X}° + [Ao sin (09) + Y]2}42 (10.6-3«) 


Ao sin (09) + Y 
— asin 10.6-3! 
© = T,(X, Y) = tan [4 coe (0,1 | (10.6-3h) 


Inverse transformations are: 
X= Ty (R, ©) = RB cos (Q) — Ao cos (Mo) (10.0-4a) 
yj) (RO) R sin (QO) — elo sin (09) (10.6-4h) 
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The functional dependence on ¢ has been suppressed in writing (10.6-3) and 
(10.6-4) with the implied understanding (hat the quantities X,Y. R, and © are 
random variables defined from the respective processes al time ¢ 


Probability Density of The Envelope 


From (8.6-15), processes X(t) and Y(t) are statistically independent (at the same 
times #) because they are gaussian and uncorrelated, The joint density of random 
variables X and Y is, therefore, 


eres yly2a2 


Sxl ¥) = ; (10.6-5) 


2no 


From (5.4-4) the jacobian of the transformations (10.6-4) is readily found (o 
be R. We next apply (5.4-6) to obtain the joint density of random variables R 
and ©: 
,  {) _ rr u 2 2 
Sr. ol" 9) = ana? exp \— F753 {r? — 2rAg cos (0 — Oo) + Ag) (10.6-6) 


The density of R alone is obtained by integrating over all valucs of ©: 


2a 
Silt) = i Sr.alt, 0) a0 


_ ure =(r2 + Agtyi2a2 ! rAg cas (A= Ona? 
re ven 0 ave? dQ (10.6-7) 


The integral is known to equal the modified Bessel function of order zero 


1 ln 
1o(B) = = | eh eos lt) (10.6-8) 
0 
Thus, 
u(r rA 
Sr(r) = me ri Se eens annt (10.6-9) 


which is known as the Rice probability density. 

Equation (10.6-9) is our principal result; it is the density of the envelope R(t) 
at any time ¢, Figure 10.6-1 illustrates the behavior of (10.6-9). For A,/o = 0, the 
case of no signal, the density is Rayleigh. For Ao/o large the density becomes 
gaussian. To show this last fact we note that 


ef 
Io(B) © >I 10.6- 
off om p (10.6-10) 
so for rAg/o? large 
oe r —(r — Ay)? 
Jxlr) © u(r) anA, a? exp [== | (10.6-11) 


| 
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06 


1.2 


0 2 4 6 
tle 
Figure 10.6-1 Probubility densities of the envelope of a sinusoidal signal (amplitude Ag) plus noise 
(power o?) for various ratios Ag/o. 


This function peaks for r near Ag, and since Ag > a, the most significant values 
of r exist only near Ag. Therefore, with r = Ag (10.6-11) becomes 


7 Ao)2/262 


Salt) & Tint (10.6-12) 


which is a gaussian function with mean A, and variance o?. 
Although difficult to derive, the mean and variance of R as found from 
(10.6-9) are known (Appendix F). 


Probability Density of Phase 


The density of the phase © of (10.6-2) derives by integrating (10.6-6) over all 
values of R. We shall leave the detailed steps for the reader us an =xercise 
(Problem 10-16), The procedure is to first complete the square in r in the expo- 
nent, and, after a suitable variable change, integrate the sum of two terms. The 
result becomes (Middleton, 1960, p. 417) 


So(0) = (1/2) exp (— Aj/20?) 
Fe Ag cos (0 ~ 05) an [= sin? (0 — “2 


J2n0 207 


psec ate) (10.6-13) 
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£64) 


Om ty= Ind 


Figure 10.6-2 Probability density function of the phase of the sum of a sinusoidal signal and gaussian 
noise, Curves are plotied for u signal phase of Uy = 3/4. 


where the function F(-) is given by (B-3). Figure 10,6-2 illustrates the behavior of 
So) for various values of Ay/a when 0) = 32/4. 

For noise only, which is the case of Ao/o = 0, Figure 10,6-2 shows that the 
density of © is uniform on (0, 2m). As A/a becomes large the density approaches 
an impulse function located at the signal's phase (at 0 = Oo). Thus, 


lim [fy(0)} = 6(0 — 09) (10.6-14) 


dole am 


(Problem 10-17). 


10.7 RADAR DETECTION USING A SINGLE OBSERVATION 


Radar can be used to detect the presence (and distance) of a nearby object (called 
the radar target). A representative problem might be to detect the presence ofan 
aircraft approaching an airport, Here the airport's radar radiates a pulse of radio 
frequency (RF) energy. The pulse propagates outward until it strikes the target 
(aircraft), whereupon some of the energy is reflected back toward the radar, The 
larget’s presence can be detected at the radar simply by detecting the presence of 
the reflected RF pulse. Once the received pulse is detected the delay between the 
lime of the radiated pulse and the received pulse is proportional to the larget’s 
distance from the radar, After a sufficient time interval (called the pulse repetition 
frequency, or PRF, interval, chosen for the most distant detection of interest) the 
radar transmits another RF pulse and the entire “detection” process is repeated, 
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Received RF 


pulse plus 
noise 
(Envelape = R) 


Envelope 
detector 


Wem g(R) 


Ww, 
‘Dhreshotd 


Figure 10.7-1 Simple radar detection network. 


A straightforward implementation within the radar receiver to achieve de(ec- 
During any PRF interval noise is always being 
If-penerated noise). A reflected pulse is 
esent. The envelope detector pro- 


tion is depicted in Figure 10.7-1. 
received (mainly duc fo the radar's OWnh se 
received with this noise only when a target is pr 
duces an output H/(r) that is some monotonic function g(+) of the envelope R(t) of 
the received signal-plus-noise waveform. The first-order probability density func- 
lion of R(t) was developed in the preceding Section 10.6. On the average R(t), and 
therefore I(t), with a target present will be larger than R(t) when only noise is 
being reecived. A suitable detection logic compares W(t) loa threshold Wy. if 
W(t) > Wp the receiver decides that a target is present; if WN) s MW. it assumes 
only noise is being received. These tests amount (0 determining when D > 0 in 
igure 10,7-1; when D> (nw target is declared to be present. 

On the average the detection logic is valid. On any one PRE interval, 
however, it is possible for the recciver to make mistakes. For example, if no target 
is truly present it may occur that noise could become large enough at some time 
to make H(t) execed Hy and cause it false detection; this type of detection is 
called a false alarm, The probability of a false alarm, denoted by P;,.is 


Pia = | fal) dw (10.7-1) 
Jy 


where fav) is the probability density of "(4 given that (here is no Girgel present 
Generally, a radar wants P,, to be sinall. 

Another type of error occurs when a target is 
suely as to cancel ifs effect during the signal's duration and force W(t) < S¥,. The 
radar usually is designed such that the probability of this event, called the prob- 
ahility of a miss, 1s small; it equals one minus the detection probability, denoted 


by Py. given by 


actually present but noise is 


P= | fiw) dw (10,7-2) 
Wr 


Here fir) is the probability density of (1) when a target is present. 

In most radars Py and Py, arc parameters of greaicst importance, Wp is 
usually chosen to give a prescribed value of P,,. Pq then depends on the ampli- 
tude of the target's returned signal. In this section we shall develop expressions 
for P,, and Py when the radar makes detection decisions based on a single obser- 
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vation (uses only one PRF interval). Our results can be extended to multiple 
observations bul the details are complicated and we only refer the reader to the 
literature (Difranco and Rubin, 1968). 


False Alarm Probability and Threshold 
When there is no target only noise is present al the input to the envelope detec- 
tor. From (10.6-9) the density of the envelope of the noise is 


Pl!) = “ re Re (107-3) 


x5 ; . . 
where a? is the power in the input noise. Because the detector characteristic g(R) 
is assumed monotonic, there is an equivalent threshold Ry on R that is related to 


W,. by 
Wr = (Ry) (10.7-4) 
Ry =g7 (Wy) (10.7-5) 


where g7!(+) is the inverse function of g(-). We may then compute P,, from the 
cnvelope as follows: 


Py = [10 dw = [“rin dr 


vy Jha 
Fe iz Sy ere de wg nO (10.7-6) 
Thus, 
p\yie 
Rr= {20 In (=) (10.7-7) 
and 


; 1 \y ue 
W, = o| {26 In (+) | (10.7-8) 


where In (-) represents the natural logarithm. 
Equation (10.7-8) gives the threshold HW, that is to be used to realize a speci- 


ficd value of P,, when the noise power level is o? at the detcctor’s input. 


Example 10.7-1 A radar recciver uses a square-law envelope detector defined 
by W = 3R?. We find what threshold is required when noise power al the 
detector’s input is a? = 0,025 Wand Py, = 107% is required. From (10.7-8) 


W, = s 20.025) In (=) | x 2.07 V 


eae ee 


rps ener BALE SOS aa em ne 


IER 


bias 
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Detection Probability 


When a target signal is present the density of the received waveform’s envelope is 
given by (10.6-9), Again using the idea of an equivalent threshold Ry on the 
envelope R we expand (10.7-2) to get 


r= | Filer) dw = | fal) dr 
Wy Kr 


~ 


a r ra 7 

= | 5 1 2 et Aorre? Gp 
SITY Td G 
20" Wn (/P,,) 


v 


(10.7-9) 


0 2 4 o x MW 12 4 16 1k 
IO tog (13/207), dB 
Figure 10.7-2 Radar detection probabilities for various false alurm probabilities when delection is 
based on a single observation, [Adupted fram Burton (1964) with permission] 
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where 


7 


O(a, B) = | ELyladye 87 27? de (10.7-10) 
Mn 

is called Marcum's Q-function (Marcum, 1950, 1960). Figure 10.7-2 ilustrates 2, 
for various values of 42/207 with 7, us a paranicter, Generally, (he sonaller yy is 
required to be the larger is the necessary signal strength to achieve a given value 
of Py. 

When P,, is small while Py is relatively Jarge so that the threshold I) is large 
and signal strength is relatively large, (he approximation of (10.6-12) can be used 


in (10.7-9) to obtain 
Ao 1 
Pye ‘ae ~ p In (+)] (10.7-11) 


where f(-) is given by (B-3). 


Example 10.7-2 We find the value of Py in a receiver having Py, = 10°'" 
when the received signal-to-noise power ratio at the detector’s input is 
16.0 dB. Here 42/20? = 39.811 (16 dB). Thus, (Ag/o) — /2 In (1/P;,) & 2.137, 
From ‘Table BL and (107-11), Py & F(2.137) © 0.9837 or 98.37%, which is in 
agreement with Figure 10.7-2. 


PROBLEMS 


10-1 Show that (a) the time-averaged autocorrelation function of Syq(), as given 
by (10.1-2) is 

Rat) = Alaa b Ryg(OJ cos (ay 0) 
if X(t) is a zero-mean process, and (b) the power spectrum is 
2 
{ 


wid 


A <3 : ee 
[Ota ~ ay) ber Evry) ] AlSy alo mg) FE Syyltr bog) ] 


Santo) aes 2 
where Syy(w) is the power spectrum of X(1). 

10-2 Define transmitter efficiency yay, in an amplitude modulation communica- 
tion system as the ratio of transmitted power due to the message to the total 
power. For a zero-mean stationary random message show that 


ees Ryx(O) Ye Sy) do Xt) 
lam Ab + Ryy(0) 202 + [%,, Syx(o) dO 42 VAD 
ot A*(t) 


where Ryy(t) and Syy(w) are the autocorrelation function and power spectrum, 
respectively, of the message V(¢). 


er 
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10-3 Define crest factor K,, fora zero-mean, bounded, random signal by 
K2, = (X(t) Va! * () 


If no overmodidation is to occur, such that | 
signal of an amplitude modulation system, show ¢ 


(Problem 10-2) is 


X(t) max & Ao 10 the transmitted 
hat the transmitter efficiency 


Man ST RE 


What is the maximum efficiency for a message X(t) = Ag COS (Om! + ©,,), where 


A,, and c,, are constants while ©,, isa random variable uniform on (0, 2n)? 
10-4 Use (10.13), (10,1-4), and (10.t-14) to show that the input signal-Lo-noise 


power ratio at the envelope detector of Figure LO.1-1 is 


— 


(=) SAt) nG2,{ AR + X70) 


RGU nlCo 
NiJam NAD 


o show that (10.1-15) ean be written in the form 


7 
eal = 2nan\ — 
Gs AM ANVNG/ a 


where tay is defined in Problem 10-2. 
10-5 In an AM broadcast system the tolal average transmitted power is | kW. 
The channel gain is Gen = 3./2(107*). Average noise power at the envelope 
detector's input is 1075 W and the output signal-to-noise power! ratio of the 
receiver is 180 (or 22.55 dB). 

(a) What is the average signal power at the in 

(b) Find (S/N dam \ 

(c) What is the transmitter’s efficiency ? 

(Hint: Use results of Problem 40-4.) 
10-6 When the message in an FM system is a sinusoid, such as x(1) = Am COS 
(a, t) where An > QO and w,, are canslints, modulation index Bryst is defined by 
fly, = ACOH yy 

(a) Write an expressi 
waveform in terms of frst. 

(hb). What is the approximate 


Aw is large relative to Om? 
(c) For the specilic waveform x(t) = 0.1 cos (J0%1), what are Byyy and the 


transnnitter’s Const Kyng if the approximate bandwidth is to be 200 kz? 

10-7 Find an expression for the autocorrelation function of Sya(O. as given by 
(10.2-1), when X(t) isa gaussian, zero-mean process. Formulate the expression in 
terms of the correlation coefficient and variance of the proccss 


Mo Mee 


Use this result t 


put to the envelope detector? 


on for the instantancous frequency (rad/s) of the iM 


bandwidth of the FM signal in terms of Buss if 


T(t) = Kens | X(t) de 


i 
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CHint: Note that the expectation involving (1) leads to a characteristic 
function.) 
10-8 In an FM system the transmitted signal has 10 kW of average power and a 
bandwidth of approximately 150 kHz when a random message with a crest factor 
of a8 used (Problem 10-3). The signal passes over a channel for which Gy, = 
107% and 79/2 = 5(107 19)/3. ‘d 

(a) Find the signal and noise average powers and the signal-to-noise ratio at 
the receiver's input. 

_(b) What is the message's spectral extent if the output signal-to-noise power 
ratio of the receiver is found to be 25,000? 
10-9 Let Hylw) = KAM + Jo) and H4(o) = tj i 

a = t/jw in (103-3 

and W, > Oare constants. : 4 aise aie 

(a) Are there any values of K, and/or 1, that wi ake y 
Pe eae t Jor 1%, that will make the loop of Figure 
2 Ba If ia = ee K,= a find the loop's output noise power if white noise 

ower density \9/2 = 10°* W/Hz is applied at the input. (Hint: i 
gral given in Problem 9-40.) Aner. eae 
10-10 Show that the transfer function of the control system of Figure 10.3-1 is 
the same as the Wiener filter of (9.2-20) if as 


Syx() 
H(w) = ANNE fodete 
ae Facts 
and 
H,(w) = eo ite 


*10-11 Show that the sum of an FM w i i 
a aveform plus noise ¢ - 
can be written in the form : eee 


R(t) cos [wot + O9 + Oem() + OM) 


where 


Ogeygt) = Kung [xo dt 
and 
R(t) = C{N.(t) + A; cos [09 + Oprg(t)]}? ae (NW) + A; sin [09 + Oralt)))?>'? 


N,(t) cos [09 + Oral] — Nelt) sin (0) + Orm(!)] 
A, + N,(t) cos [09 + Ora(t)] + N,(t) sin [09 + Opm(t)] 
x 

10-12 Assume the bandpass noise N(0) in Problem 10-11 is wide-sense stationary 


es gaussian and note that if | A,| > IN.()| and | A;| > | N,(Q] most of the time. 
nen 


y(t) = tan”! f 


NA) 
Ot) & A, cos [09 + Aa(O)] -— x sin [> + Opm(t)] 
* ' 


n 
1 
i 
4 
4 
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(a) Show that the autocorrelation function of the process Oy(t), for which 
Ott) is a saniple function, is 


ite 
Revels 6+ 1) = “i Raab 60 {ke | X(€) ue} | 
i 1 


+a Raal)A sin {kn i. xO) ach 
t t 


where Ry,y,(t) and Ry,w,(t) are the correlation functions of N,(‘) and N,(f), and 
the expectations are with respect to the message process X(t) assumed statistically 
independent of the noises. (f/int; Use the results of Section 8.6.) 

(b) If noises N.(t) and N,(t) are broadband relative to the FM signal, justify 
that 


] 
Ronarlts + 1) = Tr Ryn lt) = Reyodt) 
i 


(c) If the message process varics slowly enough for values of t that are impor- 
(ant lo Ry.w,(t) such that 


tte 
Kaen | X(E) do & Kyng X(T 


is valid, show that (he expression of part (a) reduces to 


2p2 22 
ox Kinet 


I 
Royal t+ Tt) & a exp [EE | Ry.w(t) 


if X(t) is a zero-mean, wide-sense stationary gaussian message of power o. (/int: 
Make use of characteristic functions.) 
10-13 In Example 10.4-1 let ¢ = 4 instead of 1/,/2 and recompute the loop's 
output noise power N,. Compare the result with that of (10.4-25). Is there any 
improvement over the case where ( = 1/22 (Hint: Make use of the integral 
given in Problem 9-40.) 
10-14 Assume white noise is added to an FM signal and the sum is applied to a 
phase-locked loop for message recovery. Thus, Sy,~(w) = Mo in (10.4-23). 
(a) Hf 
iia W, Ws(Wa + jo) 
WW, + jax, + jo) 
where }¥,, W,, and WW, are positive constants, find an expression for the power 
contained in the noise part of (10.4-23). 

(b) Assume the loop is designed so that W, = 2w9, W2 = wo/5, and W, = 
w2/SK, where K = Kpky and Wo is called the loop's crossover frequency (rad/s); it 
is the frequency where |KH,(w)/jw| = 1. Evaluate the result found in part («) 
when wy equals the message’s spectral extent Wy. 
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(c) If K is very large, to what does (he evaluation of part (b) approach? 
[Hint Use the known integral 
! 1 (by cot — bya? + by) daw 
3 On Jug ag® + (a? = Qty aot + (a3 — 2ayayhw? + aj 


_ Ag by = Az by = (ag 442/43) 
Qaog(tty 3 — 42) 


where (ig, 4, 42, 43, Oo, by, and by are constants and ay 2? + ad? +a, 4 + dy 
has no roots in the lower half-plane when A = w + jo (Thomas, 1969, p. 249).] 
10-15 A sample function of a semirandom binary process is to be passed through 
a lowpass filter with transfer function H(w) = W/W, + jo) where 1, is Us 3-dB 
bandwidth. If the rise and fall times of the pulses in the output waveform are not 
to exceed 5% of the pulse duration 7, what minimum value of JY, is required? 
(Hint: Assume the input waveform has been at level — -l for many pulse intervals 
and suddenly makes a transition to level 4; determine rise time as that required 
for the output (o rise from — A to 0.94.) 

*10-16 Carry out the steps suggested in the text and show that (10.6-13) derives 
from (10.6-6). 

*TO-ET AD Ag/a--> ce in (10.6-13) show Uhat (10,6-14) is (ruc, 

10-18 A radar receiver uses a finear envelope detector where Wo = R&R, Find an 
expression for false alarm: probability @), in terms of Wy, the threshold voltage 
level. 

10-19 Work Problem 10-18 for a square-law detector defined by W = KR?, 
where K > Ois a constant. 

10-20 A radar uses a linear envelope detector defined by IY” = R/4. The threshold 
voltage is H4 = 0.7 volt. Measurements show that 2, = 4(1077) What is the 
noise power at the envelope detector’s input? 

10-21 Work Problem 10-20) for a square-law detector with characteristic 
W = R3/4, 

10-22 False alarm probability is 107" in a radar that must have a detection 
probability of 0.9901. When target is present what signal-to-noise power ratio is 
necessary at (he envelope detector’s input? (Hine: Assume (107-11) applies.) 

*10-23 A radar receiver as shown in Figure 10.7-f uses a square-law detector 
defined by HW = KR? where K > (is a constant. Find an expression for the prob- 
ability density of HY. : 

*10-24 A radar receiver uses a binary detection logic biased on observing N PRE 
intervals (avulfiple observations). Tf the observations in the N intervals are sta- 
Uistically independent and the detection and false alarm probabilities on any one 
observation are Py, and P,,,, respectively, find Py and P,, that correspond to an 
overall detection logic based on obliining at least a detections in N intervals, 
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| : A simpler form of (A-1) is often applicable to many practical situations. If . 
APPENDIX + = —00, x, = 0, and (x) is arbitrary except (hat it is continuous at x = Xo, ‘ 
sae ee ren 
it | Hlx)5(x — Xo) dx = (Xo) A-2 f: 
| REVIEW OF THE IMPULSE FUNCTION -° . ces 
i A useful fact that is easily obtained from (A-1) is ah 
Gs 
i x ay 
{ S(E — Xo) dé = u(x — Xo) (A-3) re 
li or, equivalently 
du(x) ; 
‘ rato a (A-4) 
|i: where tx) is the unit-step function defined by 
Gis l O<N 
: 0 «<0 hee) 


The impulse function can be generalized to N-dimensional space (Korn and 
Korn, 1961, p. 745). If we assume a cartesian coordinate system with axes oy, $2, 
ie Ey, and pistes ai &,, 06, Ey) that is continuous at the point (€, = x1, 

2 Xgyeees Oy = Xy) then an N-dimensional impulse function 6 i 
Sincabs pulse function (E,. Sayers Ey) is 


eee 
aa 
DR Dee 


eee 
Lemeeona ee 


H 
1 
i There are several ways of defining what is known as the impulse function Soe fi 
a (Papoulis, 1962) denoted d(x). The mast mathematically sound approach is to ie __ fee Ea vees EyO(Sy — Xe Sn Xaver Ey — Xy) deere dey 
< define 3(x) on the basis of its integral property. If (x) is any arbitrary function of . 
a xyt xy <X2 are two values of x, and Xo IS the point of “ occurrence” of the = P(X yy Xgy cers Xn) (A-6) 
| i impulse, then 3(x) sittisfies (Korn and Korn, 1961, p. 742) Of special interest is the two-dimensional case; it is known that d(¢,, ¢2) can be 
| 0 xa<Xo OF XO SN written as (Bracewell, 1965, p. 85) 
' a ; (Ex. 2) = (E512) (A-7) 
fi = : : WS XQ <? 
| | : 2 [xa + xo} ea aa so (A-6) becomes 
i a 
\ h(NO(X - Xo) dx = 4 | 7 (A-1) a a 
g i Ne ; xd) Xo =X Ey. Ex)O(E, — XyVH(Eg — Xa) dE, doa = HCN. Xa) (A-8) 
Ns. -m J-@ 
1 
i ; By using (A-7) with an appropriate choice of f(,, €2) we readily find 
I s fn tas Xt , i that, 
| 2 Wo) Se : for N = 2,(A-6) can be written as oe z ; | 
a: It can be shown, using (A-1), that d(x) behaves as a function having even am le ieee qied : 
a symmetry, an area of unity, a vanishingly small “duration.” and an infinite ae eee G1 Xos 62 — Yo) 61 dea \ 
J: “amplitude ” (Pecbles, 1976, pp. 34-35). ’ x | 
i = | SEs — yo) dE, | BCE, ~ Xo) dE | 
i -~ “a 
t 1 ‘The fonction is also assumed to have bounded variation in the ncighborhood of x = Xo (see frat: = u(x — Xu) — Jo) (A-9) 


note, page 321). 
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If u(x — xy)(y — yo) is interpreted as a dvo-dimensional unit-step fiction 
WX — No, J yy), We have 


u(x — Xo Y= Yo) _ 


ax dy = (x — Xo, ¥ = Yo) (A-10) 

where 
O(N — Xg, ¥ — Yo) = O(x — Xo)5(y — Yo) (A-11) 
W(X — Xo, Y= Yo) = U(X — Xo)u(y — Yo) (A-12) 


APPENDIX 


B 
GAUSSIAN DISTRIBUTION FUNCTION 


The general gaussian or normal probability density and distribution functions 
are: 


oT aN at 


fx(X) = Jinat (B-1) 
FyQx) = | "flO dé = r(=*s) | (B-2) 
a ‘ 


where —0O < ty < 0, 0 < cy are constants and F(-) is the “normalized” dis- 
tribution function for ay = Qand oy = 1; that is 


re» [’ Ig PP ae (B-3) 


-w 2n 


F(x) is listed in the following table. When ay #0 and oy ¥ 1, Fy(x) can be found 
from I(x) by use of (B-2). For negative values of x, use 


F(—x) = | — F(x) (B-4) 
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‘Table Bel Values of F(x) for 0 ox s 3.89 in steps of O.0F 


SE an 


x OO 

oO. S000 
O14 SAS 
0.2 S793 
0.3 .@179 
O4 4584 
OS 915 
0.6 .7287 
0.7 7580 
0.8  .7881 
0.9 = 8159 
10 843 
1.10 8643 
1.2 BRAY 
13 9032 
14 9192 
4.5 9432 
1.6 9482 
1.70 OS id 
18 9641 
19 9713 
200 9772 
21 YR 
2.2 986) 
2A 983 
2.4 9918 
2.8 99M 
2 MOSS 
27 NOS 
28 9974 
2.9 998l 
AQ Y9RT 
aml 2990 
220 999K 
4a 9998 
Aa 9997 
4S 999K 
1.6 9998 
7 9999 


AR 9999 


at 


S010 
SAN 
5832 
217 
6591 
6980 
7294 
1611 
7910 
8186 
R438 
8665 
BROOD 
9049 
9207 
9345 
YAGI 
2864 
9649 
YY 
9778 
9826 
86d 
9896 
49920 
9940 
8S 
6 
975 
9982 
99R7 
9994 
2993 
9995 
497 
Y9ORK 
9999 
9999 
9999 


2 


SO80 
SATS 
S871 
6255 
6628 
6985S 
7324 
7642 
£1939 
8212 
846) 
8686 
RBRS 
066 
9222 
9457 
9474 
9573 
686 
726 
O78) 
9840 
ROS 
D898 
9922 
94S 
986 
67 
976 
9982 
9RT 
999) 
2994 
9996 
997 
299K 
9999 
9999 
9999 


03 O4 OS 06 07 AS 9 


i 


8120 S1G0 S199 5249 8279 SAY ASO 
S17 S8S7) S596 SOM 5675 S74 STS 
S910 5948 —-.S9R7 6026 6064 103 tal 
6293 GIRL GIG 6406 6443 6480 GS17 
6664 6700 6736 6772 6808 6844 6879 
"20197084 .70RK 7123 7157 7190 7224 
13877389 «.7422—«.7454 .1486 7547 .7849 
16737704 .7734——.7764 .7794 .7823 7852 
7967 .7995 8023-8051 8078 8106 R133 
R238 8264 8289-8315 8340 8365 8389 
R485 8508 «= R531 «BS54 8577 8599 8621 
RIOR «872987498770 8790 8810 BRIO 
ROOT = 8925 R944 —R9G2 8980 8997 015 
9082-9099 ONS IST 9547 9162 177 
9216 25192659279 9292 9306 OMY 
9370 938293949406 9418 9429 441 
948d ADS LOSOS STS 9525 9535 9545 
9582 .989L 9599 9608 E16 9625 M63) 
9664 9671 9678 —.9GRA 093 9699 9706 
9702 97K 9744 YTSO 9756 9761 9767 
9788 9798 979K BOI 9808 9812 R17 
QRI4 98IR 9RAD_—-98AG 9850 9054 RST 
O8TL «—-OKTS = ORTR—ORBI 9884 98R7 9890 
990i 99049906 9909 9911 913 9916 
99289927, 9929 YOM 9932 9934 2936 
9943 9948 9946 99AR 9949 Y9SI 99582 
90879989 .9960 9961 9962 9963 9964 
900899699970 99TH 9972 97S 974 
9977, 997799789979 9979 99RK0 A981 
OOK} =—.9984-—99K4— 9985 9985 9986 9986 
ORS 99KR-— 99RD ORI 9989 9990 9990 
999199929992 .9992 9992 9993 999) 
999d 9908 99069994 9995 9995 9995 
9996 999699969996 9996 9996 9997 
9997 907 997 9997 9997 9998 YOR 
9998 999K —.99DK_ 999K 9998 999K 2998 
9999 9999-9999 .9999 9999 9999 9999 
9999 =—.9999- 99999999 9999 9999 9999 
999999999999, 9999 I 000 1.0000 1.0000 


a) 8) ee 
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C 


TRIGONOMETRIC IDENTITIES 


cos (x + )’) = cos (x) cos (y) F sin (x) sin (y) 


sin (x + )) = sin (x) cos (1) 4:-cos (x) sin ()) 


nt 
cos (x + *) = Fsin (x) 


sin sie = + 
Xia = -+cos (x) 


cos (2x) = cos? (x) — sin? (x) 
sin (2x) = 2 sin (x) cos (x) 
2 cos (x) = e/* + e7* 
2) sin (x) = e/* — 7 
2 cos (x) cos (y) = cos (x — 3") + cos (x + 9) 
2 sin (x) sin (y) = cos (x — y) — cos (x + 9) 
2 sin (x) cos (y) = sin (x — y) + sin (x + )) 
2 cos? (x) = 1 + cos (2x) 
2 sin? (x) = | — cos (2x) 


4 cos? (x) = 3 cos (x) + cos (3x) 


USEFUL MATHEMATICAL QUANTITIES 


(C-1) 
(C-2) 


(C-3) 


(C-4) 


(C-5) 
(C-6) 
(C-7) 
(C-8) 
(C-9) 
(C-10) 
(C-H1) 
(C-12) 
(C-13) 
(C-14) 


‘5S 


j 
j 
4 
4 
‘ 
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4 sin? (x) = 3 sin (x) — sin (3x) 

8 cost (x) = 3 -+ 4 cos (2x) + cos (4x) 

8 sin? (xy) = 3 -4 cos (2x) + cos (4x) 
A cos (x) — B sin (x) = R cos (x + 0) 


where 
R= /A? +B 
0 = tan! (B/A) 
= R cos (0) 


B= R sin (0) 


INDEFINITE INTEGRALS 


Rational Algebraic Functions 
(a + bx)! 


b(n + 1) oh 


[i + bx) dx = 


dx 
JeBe-pe Ja -F bx| 


| dx -| id 
(a+ bx)” (n — Wb(a + bx)! i 


fot ee (GES) b? < 4ac 
ctbx tax? /4ae — b? /4ac — b? 


1 o 2ax +b Jb? — 4ac 52 > da 

Pe aa ac 
Jb? -~4ac |2ax +b + fb? — 4ac 

a5 : 

= =f 

2ax + b e a 
x dx ites fbs-bel= = dx 

c+bx tax? 2a c + bx + ax? 


bx 
-1 [OX 
a eae oe (=) 


Jeraeear 
Jarre 
ied l 
Jn 


ao n (a? +x?) 


(C15) 
(C-16) 
(el) 
(C-18) 


(C-191) 
(C-19b) 
(C-19¢) 
(C-19d) 


(C-20) 


(C-21) 


(C-22) 


(C-23) 


(C-24) 


(C-25) 


(C-26) 


(C-27) 
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x | x 
= - ran a C08 
lan x2? aa? + x?) "ae () 
xadyx —! 
= C29 
[ats +x? Aa? 4 x?) | 
x — —Xx I x 
= yet tan |= C-30 
Jair +4? Yar tx it Dare (:) ( 
x 3x 3 xX 
ja er a ln = Oi 
[ots (a +x ane Pe + x?) Yee + x?) Chak (:) oo ! 
-—Xx xX i -[{X 
= ae ae ae ti = “32 
, ap date delet ae) tee (:) era) 
dln ee 5x 3 N 
? y = tt a = C-33 
ln x?) ey dae Te)? Bla? tx) ‘ ca (:} ( 
x 5x : 5x ‘ eer (: 
(a? + a + Gra? + 2p * aaa 42) * 160%(a? # x?) 160? a 
(C-34) 
[at =X x ing Xx { an 1 (8 
prea ts az . Se ene Te * 
wd ey ba? +x) " 48 (@ exe loat(a? + x? 4)" Toa? a 


(C-35) 
a ate 
(+ er (e+e) Mate orate) ia \a 


(C-36) 


dx x? - ax /2 +7 ax./2 : 
itenaian eth) ia (2) =m 


a 


vt —ax/2+0 


ae 


wee 4a./2 


2x (S + ax,/2 + a? 


| ( 
+ ——= tan”! ~ 
x? = ani/2 + “) 2a,/2 a — xt 


Trigonometric Functions 


| cos (x) dx = sin (x) (C-39) 


iE cos (x) dx = cos (x) + x sin (x) (C-40) 
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FINITE SERIES 


[«? cos (x) dx = 2x cos (XN) + (x? — 2) sin (x) (C-41) 
z N N(N 4: 1) 
Yas : 
[xn (x) dw sz cas (x) (C’-42) av 2 (C-55) 
: 3 2 = NON + DQN + 1) eat 
yosin (x) dy oo sin (vw) =v cas (y) (C-43) net 6 
’ Us N?7(N | 1)? 
2 Vout wee . 
ae 4 (C-57) 
| sin (\) dy = 2x sin (x) = (x? = 2) cos (x) (C-44) : 
> die xNity 
ony ne | (C-58) 
exponential Punctions ; N! ated is 
tx L 1 N — nl xy = (xX + y. (C-59 
fer dx=— areal or complex (C-45) ve ms i) 
a ; 
Yi elton) sin {(N i ite +9720 een 
: a= sin _ 
[oe dx = ak = =| a real or complex (C-46) : #/2) 
aoa ; r N N Ni F 
woody 2 xn) =X mai? (C-61) 
[sve dx = ak; - a + | a real or complex (C-47) 
as th 
0 3x2 xX 6 
| xton dx = ab - =. + a - 5 a real or complex (C-48) 
aoa wooa 
fe sin (x) dx = —— (a sin (x) — cos (x)] (C-49) 
a+ | 
fe cos (v) dv = ra fa cos (x) -F sin (v)] (C-50) 
q 


DEFINITE INTEGRALS 


| : ge MMS IN ae sian ghia a>d (C-51) 
ore a 
i vreoV dx = Jn/4 (C-52) ‘ 
0 
~ "sin (x) n - 
Sa (x) dx = — dx =o (C-53) 
o a x 2 


[se (x) dx = 1/2 (C-54) 
a 
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D 
REVIEW OF FOURIER TRANSFORMS 


The Fourler transformt or spectrum X(w) of a signal x(t) is given by 


oe eo 

X(w) = | x(t)e 4"! dt (D-1) 
-«@ 

The inverse Fourier transform allows the recovery of x(t) from ils spectrum X (cv). 

itis given by 


x(1) = ae | X(w)e" leo (1D-2) 
22 Jaw 
Together, (D-1) and (D-2) form a Fourier transform pair. Extensive tables of 
transform pairs exist (Campbell and Fuster, 1948). A transform pair is often sym- 
bolized by use of a double-ended arrow: 


x(t) + X(w) (D-3) 


The Fourier transform X(w) is valid for real or complex signals, and, in 
general, is a complex function of w even for real signals x(t). X(w) describes the 
relative complex voltages (amplitudes and phases) as a function of w that are 
present in a waveform x(t). From (1-1), we see that the unit of X(w) is volls por 
hertz if x(t) is a voltage-time waveform. Thus, X(w) can be considered as the 
density of voltage in x(t) as a function of angular frequency a. 


t Named for the great French mathematician and physicist Baron Jean Baptiste Joseph Fourier 
(1768-1830). 
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EXISTENCE 


Conditions (hat guarantee the existence of the Fourier ¢ransform of a waveform 
N() are: 


1. that x(t) be bounded with at most a finite number of maxima and minima and 
a finite number of discontinuities in any finite (ime interval,t and 


2: [. Ix()| dt < o . (D-4) 


“oO ‘ 


‘These conditions are only sufficient for X(w) lo exist; (hey are not aecessary, 
Many signals of practical interest do not satisfy these conditions but do have 
transforms, Examples are: the unit-impulse function d(f) that has the transform 
X(w) = 1; and the unit-step function u(t), defined by uf) = 1 for O<¢ and 
u(t) = 0 for ¢ < 0, that has the transform X(w) = n0(w) + (1/jw). 


PROPERTIES 


A number of extremely useful properties of Fourier transforms may be stated. We 
give these without proofs since the proofs may readily be found in the lilerature 
(Peebles, 1976, p. 29; Papoulis, 1962, p. 14). In these properties, we assume the 
Fourier transform of some signal x(t) is X(w), while the notation X,(w) implies 
the transform of a signal x,(t) with n= 1, 2,..., N. 


Linearity 


For constants a, ((hat may be complex): 


N N ‘ 
x(t) = ¥ axe Y a, X,(w) = X(w) (D-5) 
nw} nay 
Time and Frequency Shifting 
With fy and (wg real constants: 
x(t — fo) + X (we /"9 (1D-6) 
(Nel or N( = wy) (19-7) 


t These are known as the Dirichlet conditions, after the German mathematician Peter Gustov 
Lejeune Dirichlet (1805-1859). A signal satisfying them is said to have bounded variation (Thomas, 
1969, p. 579). , 


bt : i ‘ ‘ 
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aa Scaling Parseval’st Theorem 
yy With « a real constant: o 1 f? 
i \ * (D-8) x#(t)x,(t) dt = i X*w)X (eo) dew (D-20) ' 
; ‘ aes = -o -—m i 
a se tal x(2) | 
ij An alternative form occurs when x,(f) = x;(t) = x(t): 
i 
Duality 7 ese ee ; 
i. : X(t) Inx(— co) (D-9) | Jol dt= Fi | | X(e)|? den (N-21) 
i | 
Differentiation : | 
h. d"x(t) = (jeo)"X (co) (D-10) MULTIDIMENSIONAL FOURIER TRANSFORMS j 
ul. “d" : 
; er The Fourier transform X(w,, cw) of a function x(t,, 1,) of two “time variables ¢, 
j d"X(c) ; ' 
ae (—jnyix(n) a _— (D-11) and ¢, is defined as the iterated double transform. Upon Fourier transforming 
ie day X(ty, fy) first with respect to ¢, we have | 
" Integration ’ X(W4, ty) -{ X(ty, Je" dt, (D-22) 
1 X(w ~@ 
au x(t) dt aX (0)5() + xe) (D-12) : 
co ss Ja X(w 1, 2) results from Fourier transformation of X(w,, t,) with respect (o fy: 
q on 2a (" xe (D-1) ' : 
i : Oe [ Re Xo, 03) = { Xen, fe" dt, (D-23) 
4 . or 
hi Conjugation (10-14) oi a 
x x*(Qheoo N*( ~ a) oa Xo, «,) = | { Mi Ge donuts Jongey dty dty (1-24) 
xt =o Xo) ‘ie oe 
tt By use of similar logic, the two-dimensional inverse Fourier transform is 
’ Convolution | m fo 
gh : My. 03) = Ont X(ea,, ay) 4422 day, dang (1-25) 
lie xt) = | xyle)g(t = 1) dr er X(@)X (oo) = X(o) mis) CE oe Ice 
\; a ; The extension of the above procedures to an N-dimensional function is 
fi x(t) = xy(OX (ven { X (OX (wo ~ & dé = X(w) (D-17) direct; we obtain the Fourier transform pair 
4), $ SS NEPAD Qn 
Ky y < 2 ®0 
X (Wy, 16+) Oy) = { tee | X(lyy cons tye ZOtem dene dees dty  (D-26) 
Pao . “2 “2 
mihi. Correlation 
Hh a u x feagey bod Jain 
qe X(t) = | N#(t)Ny(t doe Nt(aQ)N ta) = X(e) (D-18) N(Oqs vee ty) = (2n)" ee As veeg ye days dwy (D-27) 
it MU) = Sel: -@ - 
aul? sees 
ai o*(1)N poe i X*(E)X AE + cv) dg = X(o) (Dy) 
BY XA) = XT(ON] dn Jon : t Named for M. A. Parseval. 
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PROBLEMS 


D-1 Find the Fourier transform of a pulse x(¢) defined by 


x(t) = 
aan 10 


where t > Oand A are real constants. 
D-2 Ifa signal p(s) is the product of x(t) of Problem D-1 with a cosine wave, that 
is, if 


—1/2<t<1/2 
elsewhere 


Ht) = x(1) cos (Wot + 90) 
where (vg and Oy are real constants, what is the Fourier transform of y(t)? 
D-3 Find the Fourier transform of the waveform 


A it) 
- Jt}<t 
x(t) = T 


0 Jt]>t 


where t > 0 and A are real constants. 
D-4 By direct use of (D-1), find the Fourier transform of the waveform 


__ JA cos (1/27) |th<t 
x= {4 [el >t 


where t > Oand A ure real constants. 
D-5 The waveform of Problem D-4 can be written in the form 


x(t) = A rect (¢/2t) cos (nt/2t) 
where rect (1/21) is defined by (E-2). By using (D-19), find the Fourier transform of 


* x(t). 5 


D-6 The complex form of the Fourier series of an arbitrary periodic signal y(t) of 
period T is 
y(t) a ° C, ginrniT 
ne-@ 
where the Fourier series coefficients are given by 
T2 


i 
C= 5 | yaje 22497 dt 


-T/2 
forn=0, +1, +2,.... Show that the Fourier transform of this arbitrary period- 
ic signal is 

= n2n 
Y(w)=2n YO C,d6| 0 —-—- 


ntm wy Tr 


where o(-) is the unit-impulse function of Appendix A. 
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41)-7 Prove the Fourier transform pair 
Mee on Ss nin 
Y 6 - aly » i(w -— 
neu aera i 
where ‘1 > 0 is a real constant and d(-) is the impulse function of Appendix A. 
(int: Represent the time function by a complex Fourier series as in Problem 
D-6, find the Fourier coefficients of the series, and then Fourier-transform the 
series). 
*D-8 From the expression in Problem D-7, it is readily shown that 
ue ue 4 
yy grit, 2% xy i(w - 2) 
neBow T n®—- 0 T 
Use this result (o prove that the periodic signal 


o 


w= Extn) 


aswn~mw 


comprised of repetitions in each period T of a basic waveform x(0, has the 
Fourier transform Y(w) given by 


Qn & 2 
Y(w) = = SE (Filo a ar) 
where X(c) is the Fourier transform of x(¢). By using the result of Problem D-6, 


we see that the coefficient C, of the Fourier series of y(t) is related to the Fourier 
transform of its component waveform x(r) by 


1 fn2n 
c= Ex it) 


D-9 Find the Fourier transform of the waveform 
x(t) = u(tete" 


where (+) is (he unit-step function of (A-5) and qo is a real constant 
D-10 Find the Fourier transform of a sequence of 2N +1 pulses of the form 
given in Problem D-1, where N = 0, |, 2,.... That is, find the transform of 


N 


x)= Y xt—n7) 


naoN 


with T > Oa real constant and t < T. 
D-11 Determine the Fourier transform of the signal 


Ar O<t<t 
x(t) = 
0) elsewhere 


where t > Oand A are real constants. 


: 
‘ 
. 
2 
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D-12 Show that the-inverse Fourier transform of the function 


‘ K -W<w< IV 
X(w) = 
0 elsewhere 


x(t) = (KW/n) Sa (2) 
where W > Oand K are real constants and Sa (-) is the sampling function defined 


by (1-3). 
Det) ‘The (ansfer function (iw) of a lowpass filler can be approximated by 


N 
Ko +2 ¥ K, cos (nnw/tW) —We<w<W 
Hw) = nat 


0 elsewhere 
Here Wo> 0. Ko. Kye eeee Ay are real constants and N > Qis an integer. Find the 
inverse Fourier transform A(t) of Htc) which is the impulse response of the 
network, in terms of sampling functions (sec Problem 1-12). 
1-14 Let x(t) have the Fourier transform X(«). Find the transforms of the fol- 
lowing functions in terms of \(c): 


W) = Deep Gon) — (6) Sex Lingle] = 9) = 3820 


Here dy is a real constant, 
D-15 If x(ther (co), find the inverse transforms of the following functions in 
terns of x(t): 


(a) N(w)N (an by) (b) NX (ua -- ag) oro (c) N*(—c) 1X) 
dw 


Here * represents complex conjugation and wm, is a real constant 

D-16 A voltage \(f) exists across a resistor of resistance R. Show that the real 

energy [2 expended in the resistance is 
I 


| ined 
2nk | 


[" |X(a)P> de 


where V(r) is the Fourier transform of x(t). 
1-17 fis known that 


2a 


x(Q = eval = NX(w) 
( ) a? -{- wy? 


where x > Ois a real constant, Find the Fourier transform ¥ (co) of 


(6) 
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1-18 Use the definition (A-2) of an impulse function to prove that the impulse 
has the Fourier transform 1. That is, show that 


S(t) I 
D-19 By use of vari ouri i 
as y arious Fourier transform properties, show that the following are 
(a) A + A(2n)5(w) where A is a constant 


(b) cos (a9 1) + n[5(w — a9) + 5 i 
o) + (CH) -k 9)] where wis a realc i 
1-20 Use the facts that ‘ as 


(the 7% e 
a+ jw 
and 
COS (pt) + n[d(W — Ug) + Kw + Wo)] i 


1 
where a > Oand mg are real constants, to prove that 
u(t}e~* cos (ag fa 
(a? + we — w) + (Qa) 
D-21 Prove (D-6) and (D-10). 
*“D-22 Prove (D-12). (Hint: Use (D-16).] 
1)-23 Prove (D-18). 
D-24 Vind the Fourier transform of the signal 


tntd= fo tT <ty<t, and ~t) <0, <1, 
0 elsewhere; 


where ty > 0, t, > 0, and A are real constants. 4 
ay 


wt ae Se 


Pais x(t) N{a) Notes 


ee 
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0 
TABLE OF USEFUL FOURIER TRANSFORMS 


H ad{t) ” 


2 a/2n ad{cw) 
3 u(t) nd(ta) + (1/ja) 


: { 
4 Yd(t) ~- jin (ev) 


5 rect (t/t) t Sa (wt/2) t>0 
6 (Wn) Sa (0) rect (w/2WV) W> 0 
7 tri (t/t) t Sa? (cwt/2) t>0 

& (IV fx) Sa? (Ya) tri (w/21¥) W>0 
D) lent Ind(w — ey) 

10 Ot -- t) "eee 


1 
{ 
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‘Vable E-f Fourier Vransform Pairs 
4 
I cos (i, 0) n{ (aa — cay) + (to -b aay) 


12 sin (ty 0) —jn[S(w — ay) ~— S(@ + wy)] 
| 13 u(t) COS (ey 1) ; [ Sco — arg) -b Slr 4° 049)) wa 
i 
i 14 H(t) sin (ay) ~j = [8 — wy) — Slew + eng) + 
; 2 ww 
1 
15 eat < 
mat a + jw a0 
a | 
16 u(t)te ere a>od 
17 2,0 at 2 
ure rer) a>0 
In the following table of Fourier transform pairs, we define 7 6 
oer 18 nee — a>o 
(a + jw)* 
ugesi SF (E-1) 
0 &<0 ea tt 
a? + ey? ao 
1 1él< "4 
rect (¢) = E-2 20 peel) Qn ee 
() {0 Iel> 4 (E-2) ; af2ne a>0 
sin (¢) ‘ 
Sa (€) = ——- (E-3) 
¢ 
i-|é} [el <! 3 
tri (€) = ted 
| wt Igl> ar 
Mo X(w) (IE-5) 


and let a, t, ¢, ag, and JY be real constants. 
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APPENDIX 
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SOME PROBABILITY DENSITIES 
AND DISTRIBUTIONS 


For convenience of reference we list below the probability density f(x) and dis- 
tribution function /y(x) for some well-known distributions, Where appropriate, 


we also give the mean XY, variance at,and characteristic function @ (a). 


A number of constants and functions are used as defined below: 
dy yy dy. Dy, Dy, a and p are real constants 
N is a positive integer 
6(é) = impulse function of (2.3-2) 
u(€) = unil-step function of (2.2-4) 


rect (€) = rectangular function of ((-2) 
Px) = | EXT eS dE Re (x) > 0 
0 
= gamma function 
1 ff , 
P(x, f= Elen S dé Re (a) > 0 
| (7) fl 


-incamplete pamina function 


t Re tri denotes the real part ots, 


AM 


(F-1a) 
(F-1b) 
(FF-1e) 
(F-Td) 
(Fle) 


(FAI) 


(I*-1y) 
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| x 
P(x|N) = Tarp i EINDV= 1g U2 Gy 


= chi-square probability function 


oss [ope 
uy = Pom 
Tip + { ae as 


(F-1h) 


= Pearson’s form of incomplete gamma function (Pearson, 1934) 


= P(p+1,u/p +1) 
(a+b) (* 
P(a)l(b) Jo 


I,(a, b) = 


= incomplete beta function 


F(x) = gaussian distribution of (B-3) 


. awe _(x/2)"* 
L(x) = (: ene) Ss 
BE) Xs, kI(n + k)! 


| n 
= | 0% £8 cos (10) dO 
T Jo 


= modified Bessel function of first kind of order n = 0, 1, Pee 


Qa, f) = [etotoe exp 4) dé 


2 


cant = gyro} dé 


(F-1i) 


(F-1)) 


(F-1k) 


(F-11) 


(F=f) 


The functions of (F-If) through (F-1/) and that of (F-!0 are discussed in detail in 
Abramowitz and Stegun, cditors (1964). Q(a, 2) is Marcum’s Q-function; it is 


tabulated in Marcum (1950), 


DISCRETE FUNCTIONS 


Bernoulli 

ForO<p<! 
Sy(x) = (1 — p)S(x) + pd(x — 1) 
Fy (x) = (f= p)u(x) -b p(x ~ 1) 


“ 
u" 


p 
ax = pl — p) 
@y() =[|— p -f pei 


(F-2) 
(1-3) 
(1-4) 
(F-5) 
(I*-6) 


ee aS 


TESST 


WIDaT MEP AT 


Soha bee 


reed 


ye ates 
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Binomial 


ForO<p<landN=1, 2,... 


NN 
I(x) = p> ( x ene — py"~*8(x — k) 


=9 
x (N\, Nok 
Fels) = 3 (4 pent = pitas ~ &) 
k=0 
X=WNp 


at = Np(i — p) 
@y(w) = (1 — p + pel)” 


Pascalt 


ForO<p<landN =1,2,... 


sso (871 ot = aM = 8 
k 


oy \N =I 
Fy(x) = Y (opt = ata = 
kun 
Pie 
p 
N(t — p) 
2. oe 
Oxy = p 


D(w) = prelC1 — (1 — p)ely-% 
Poisson 
Forb>0 


7) k 
Syxy= er? 2 d(x — k) 


vo k! 


ry b* 
Fpxyser? Ym ux —k) 
nao KI 


<I 
" 


b 
t=b 


,(w) = exp [b(e” — 1] 


Q 


¢ Blaise Pascal (1623-1662) was a French mathematician. 


(¥-7) 


(F-8) 


(F-9) 
(F-10) 
‘Fetl) 


(F-12) 


(F-13) 


(I-14) 


(F-15) 


(F-16) 


(F-17) 


("-18) 
(F-19) 
(F-20) 
(F-21) 
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CONTINUOUS FUNCTIONS 


Aresine 
Fora > 0 


rect (x/2a) 


Sx(x) = SY px rot 


0 -oO<x< —a 
l i x 
Fy(x) = 45+ sin”! (:) -agx<a 
< "4 a 
| usgxn<w 
v= 0 
2 
ox = 2 
Beta 
Fora>Qandb>0 
_ Ma +) A e an! ood 
Sx) = For) (u(x) — u(x — 1px? = x) 
T (a, b)u(x) x<l 
F = 
x0) { x2 
yates 
a 
gre ab 


(a + bP (a +b + 1) 
Cauchy 
For h> QOand -o <a<@ 


sa hate (b/n) 
Axx) = b? 4 (x — a)? 


1 4 c+ 
Fy(x) = oe tan”! (: ; *) 


X = is undefined 


a2 = is undefined 


(cv) = else ~ bie] 


(F-22) 


(F-24) 


(F-25) 


(F=30) 


(F-31) 


(+32) 
(F-33) 
(F-34) 


i 


ON ed 


mets 
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Chi-Square with N Degrees of Freedom 
For N = [, 2,... 


Erlang 
For N = |, 2, 


Exponential 


Fora>0 


.. and a> 


Ly(x 


DP y(co 


0 


) 


yi) | 


2° TN /2) : 


(1 — j2m)7 8? 


Sy(x) = ae” 1x) 
Fy(x) = [1 — e7*Ju(x) 


(F-35) 


(F-36) 


(F-37) 
(7-38) 
(17-39) 


(F-40) 


(F-41) 


(7-42) 


(7-43) 


(F-44) 


(F-45) 
(F-46) 


(F-47) 


(F-48) 


(17-49) 
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Gamma 


Fora>Oandb>0 


fis) = FE yy 
F(x) = (S. b= ats 
come 
i 
oe 


b 
Oxley = (- +) 


(F-50) 


(F-51!) 


(F-52) 


(F-53) 


(F-54) 


Note that if b is a positive integer the gamma densily becomes the Erlang density, 


Aiso if bh = N/2, for N = 1, 2,.. 
chi-square density. 


Gaussian-Uniyariate 


: Forb>Qand -a@ <a<o 


Sy(x) = (aby7 V2 er een antin 


Xv~a 
Py(x) = F 
xl) ( 4 


X=a 

b 
apd 
ox 5 


y(w) a eda (th]4) 


Gaussian-Bivariate 
For —00 <a, <0, —© <a, < 0, b, > 0,6, >Oand -] Sspsl 


Say xy X)) = [x7b,b,(1 a pal a 


| (x, — a,)? 
ta = mit h, 


m 2p(Xy — GiXXy = ay) i (X2 - cal 


Vib, : by 
xX,-a,_ "x, -a 
Frusibin sy) = (| aa -| 2 | ) 
Ny XQ\AE 2 : rr mT I 


yatnd as 4 the gamma density becomes the 


(F+55) 
(F-56) 
(F-57) 
(F-$8) 


(F-59) 


(F-60) 


(F-61) 


See eee ee 
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where L(x,, x;, p) is a probability function discussed extensively and graphed in 
Abramowitz and Stegun, editors (1964), p. 936. Also 


Ra, | (F-62) 
re (F-63) 
oh, = bf? (F-64) 
ot, = b,/2 (63) 
Dy, yg(ys Wg) = exp (jeayay + jeur ay — Yylwib, + 2pw,w2/b,b, + w}b2)} 
(F-66) 
Laplace 
Forb>Qand -w <a<o 
l 
fils) = 5 evens (F-67) 
Vedxma —-o<x<a 
Fy(x) = <4? F- 
x) { — Yew dena agx<o Es8) 
X=a (F-69) 
2 2 a 
oY = i (F-70) 
: pine : 
Dy(w) = b Poa (P-7) 
Log-Normal 
For -w <a<w,-w<b<w,anda>0 
1x os b)e He {x-b)-a}2/202 
fd) = SS 72 
A J2nx — bo 7 
Fy(x) = u(x — db) {a7 Dn (x — b} — a} (F-73) 
~ o 
X =b+exp (« + 5) ('-74) 
ak = [exp (a?) — 1] exp (2a + 07) (F-75) 
Rayleigh 
For -o <a<omandhb>0 
2 
Sx) = 5 (x — ae" Pix — a) (F-76) 
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F(x) = [1 = eT 87 ux — a) (F-77) 
f t a 
X=at fe (F-78) 
oe Li n) ‘cn 


Rice [Thomas (1969), Middleton (1960)] 


Fora >OQOandb>0 


an 


Iy(x) = S en? vein (Sa) (7-80) 


Fy(x) = E ad a(¢. =) ha (F-81) 
: Rocha (Re ed RO Re fs 
ran frewd(i+E).() +5 0(4)| (F-82) 


of = (2 +k?) — (XP (F-83) 
2 
= A (7-84) 


Uniform 


For-w<a<hbh<aw 


u(x — a) — ux — b) 


Jx(x) = cae rer maas (17-85) 

hon | eee ve? (F-86) 
l x2b 

X= aie (F-87) 

ay = bow (F°-88) 

Dl) (F-89) 


~ jo(b — a) 


a 


po 
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Weibull 
Fora > Qand b> 0 : 
Py(x) = abst Fortis) (F-90) 
ByQ) sf et Jute) (F-91) 
. deh ') ange 
Soe | Sake (1-92) 
A qh 
Paty ty rr holy : 
a ma LUE ee IAL ae (F-93) 
' a 


Note that if’ = 2 the Weibull density becomes a Rayleigh density. 
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A posteriori probability, 18 
A priori probability, 18 
Algebra of sets, 7 
Ambiguity function, 272 
Amplitude modulation (AM), 192, 275-279 
Antenna temperature, 231 
Aresine density, 84, 333 
Associative law of sets, 7 
Autocorrelation function: 
bound on, 153 
of complex process, 160-161, 193 
defined, 149, 152-154 
of derivative of a process, 106, 246 
Jerivatives of, 203, 246 
Fourier transform of, 177, 180, 193 


lourier transform of time average of, 177, 180 


of linear system response, 212-213 
nieasurement of, 156-158 

at product device output, 191 
properties of, 153 


relationship to power spectrum, 177, 179-181, 


193 
of stationary process, 149 
time, ISL 
time average of, 177, 179-180 


of wide-sense stulionary process, 150, 153 


Autocovarinnce function: 
of complex process, 160 
delined, 155, 160 
Fourier transform of, 196 


of wide-sense stationary process, 156, 158 


Available power gitin, 232 

Average effective noise temperature, 237 
Average elfective source temperature, 237 
Average operating noise figure, 236 
Average standard noise figure, 236 
Axioms of probability, 10 


Band-limited random process, 219 
properties of, 221-223, 224-226 
Band-limited white noise, [88 
Bandpass random process, 219 
Bandwidth: 
noise, 210, 217-218, 238 
of a power spectrums, 177-179 
rms, 177-179, 198, 201 
Bayes, Thomas, 16n. 
Bayes’ theorem, 16 
Bernoulli densily, 331 
Bernoulli, Jacob, 251. 
Bernoulli trials, 25 
Bessel function, 299, 331 
Beta density, 333 
Vinary communication system, 17 
Binary process, semirundom, 162 
Binary symmetric channel, $7 
Binomial coeMicient, 26, 33 
Binomial density function, 46, 332 
characteristic function of, 85 
mean of, §t 
variance of, 81 
Binomial distribution function, 46 
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Conditional density function, 52, 62 
with interval conditioning, 99 
with point conditioning, 96-99 
properties of, $3 


Cross-correlation function(s): 
Fourier transform of, 184, 185-186, 193 
of independent processes, 154 
of jointly wide-sense stationary processes, 150, 


Distributive law of sets, 7 
Domain, 58 
Duality principle, & 


Rivarinte gaussian density, 124, 335 
Boltzmann, Ludwig, 2270. 
Boltzmann's constant, 227 
Bound(s}: 
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on autocorrelation function, 153 
on cross-correlation function, 184, 155 
on linear system response, 210 


Bounded variation, 31Qn,, 424m 
Butterwarth filter, 243 


Cartesian product space, 22n. 
Cauchy, Augustin Louis, 61a. 
Cauchy density function, a4, MM 
Cauchy distribution function, 02 
Caueby conden variable, 61 
Cisal systens, 210 
Central limit theorem, 105-108 
Central moments, 70, 82, 120° 122 
related ty moments about origin, 82 
Certain event, 10 
Channel, 17, 276, 281 
binary symmetric, $7 
Characteristic lunction: 
of binomial density, 85 
delined, 72, 122 
of Erlang random variable, 85 
ofexponential random variable, 7273 
of gaussian random varisble(s), 8.3, 138 
joint, 122-124, 138 
marginal, 123 
moment generating properly of, 72, 123-124 
of Poisson random variable, 85 
for several random variables, 124 
for two randons variables, 122 
Chebychev's inequality, 8S 
Chebyches, Patnuty L., RSn, 
Chi-square density, 85, 331, 334 
Class af sets, 3 
Classification of process, [44 146 
Coefficient of skewness, 71 
Colored noise, 190 
Combined experiment, 22 
Combined sample space, 22 
Communication system(s): 
amplitude modulation (AM) 275--279 
binary, 17 
frequency allocations, 275-276, 280 
frequency modulation (EM), 280 .2Kd 
threeesymbhot, V3 
Commutative law of sets, 7 
Complement of scl, 6 
Complex randany process, 160-161 
(See alse Random process) 


Comples random variable, 134 


Conditional distribution function, 52, 62,96 
with interval conditioning, 99 
with point conditioning, 96-99 
properties of, 52 
Conditional expected vatue, 69, 139 
Conditional probability, 13 
Conditioning event, 55-57 
interval, 99 
point, Y% -99 
Continuous randam process, 145 
Continuous randam sequence, (46 
Continuous eiidoat variable, V7 
Convolution integral, 103, 207, 223 
Convolution of density functions, $03, 105 


Correlation: 
of independent random variables, 1t9, 134 
of orthogonal random variables, 119, 134 
of random variables, £9, 134 
of uncorrelated random variables, 119, 134 
Correlation coefficient, 121,136 
Correlation functions: 
autocorrelation, 149, 152-154, 160 16l, 
212-24) 
autocovariance, 155-156, 160-161 
cross-correlation, 150, 154, 213-214 
cross-covariance, 156 
of derivative of a process, 166, 246 
lime autocorrelation, 151 
time cross-correlation, 152 
(Sve also listings by specific types) 
Correlation integral, 268 
Correlation receiver, 269 
Countable set, 3 
Countably infinite set, 3 
Covariance: 
of independent random variables, 124 
of orthogonal random variables, 121 
of random variables, 121, 14 
of uncorrelated random variables, 121, 134 
Covariance fnnetions (sec Correlation functions) 
Covariance matrix, 127 
after finear transformation, 132 
Crest-factor, 280, 306 
Crossover frequency, 308 
Cross power, ERY 
Cross-correlation function(s): 
bounds on, 154, 155 
of complex pracess, 160-161, 193 
defined, 150, 154 
of derivative of process, fO6, JOA, 246 


1$4 
measurement of, 156-158 
of orthogonal processes, 154 
properties of, 154 
real and imaginary parts of, 199 
relationship lo cross-power spectrum, 184, 
185-186, 193 
of response of linear system, 213-214 
time, 152 
time average of, 184, 1RS-186 


Cross-covarianee finetion: 


of complex processes, 160-161 
defined, 156 
of independent pracesses, 156 


of uncorrelated processes, 156 

of wide-sensz stationary processes, 156 
Cross-power density spectrum: 

of complex processes, [93 

defined, 183 

of linear system, 217 

properties of, 184 

real and imaginary parts of, 199 
Cross-power densily spectrum: 


relationship ta cross-correlation function, 184, 


185-186, 193 


Cross-power spectrum (see Cross-power density 


spectrum) 


Cumulative probability distribution function (see 


Probability distribution function) 


Damping factor, 295 

Decibels, 240, 

Delta finetion (see Impulse function) 
De Morgan, Augustus, 7n, 

De Morgan's laws, 7, 32 


Density function (see Probability density function) 


Derivative of random process, 166, 203 
power spectrum of, 177 

Detection probability, 304-305 

Deterministic random process, 147 

Deterministic signal, (73 

Dilference of sets, 5 

Dirichlet, Peter Gustav Lejeune, 321n. 

Dirichlet conditions, 321n, 

Discrete random process, 145 

Discrete random sequence, 146 

Discrete random variable, 37, 79 

Disjoint sets, 3 

Listribution function (see Probability distribu. 

tion function) 


Effective noise temperature, 229, 233, 237 
Efficiency, 279, 305 -306 
Elementary event, 12 
Elements of a set, 4 
Umpty set, 3 
Ensemble, 143 
Ensemble average, [43 
Ensemble member, 143 
Envelope detector, 278 
Fqual sets, § 
Erpodie nindam process, 182, 1871, 
Ergodic thearem, 182 
Erlang, A. K., 85a, 
Erlang density, 334 
Erlang random variable, 85 
Event(s): 
certain, 10 
on combined sample space, 23-24 
defined, 9 
elementary, 12 
impossible, 10 
joint, 89 
mutually exclusive, 9 
pairwise independence of, 19 
probability of, 10 
statistically independent, [8-22 
lExcess available noise power, 233 
Expectation (see Expected value) 
Expected value (expectation): 
conditional, 69, 139 
ofa function of a random variable, 68, 133 
of a function of several randam variables, 
T17-119 
of random process, 143, 153, 160 
ofa random variable, 67-68, 133 
ofa sum of random variables, 148 
Experiment(s), & 
combined, 22 
independent, 25 
mathematical definition of, 8, 11-12 
Exponential density function, 49, 68, 70, 71, 
72-74, 334 
Exponential distribution function, 49, 334 


False alarm, 302 

Filter (system): 
Butterworth, 243 
gaussian, 241 
idealized, 209 210 
tuttehed, 258 -262 
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Hiller (system): 
prediclon, 263 
smoothing, 263 
Wiener, 262 -266 
Finite set, 3 
Miest-ondes stationary cundom process, bf 
Four, Maron Jean Baptiste Joseph, $20, 
Fourier series! 
cuclicients, 324, 325 
defined, 324 
Fourier transform of, 324, 325 
Fourier teansform(s): 
of wutocurrelation function, £77, £80, 193 
of cross-currelation function, 184, 185-186, 
193 
defined, 72, 173, 320 
existence, 321 
inverse, 72, 173, 320 
multidimensional, 323 
properties of, 321-323 
table of puirs of, 329 
of time-averaged autocorrelation function, 177 
of lime-averaged cross-correlation function, 
1&4 
uniqueness, 161 
Frequency-domuin unatysis, 172 
Frequeney modulation (PM), 280-284 
Fresnel integrals, 273 


Gammit density, 335 

Gamma function, 85, 330, 331 

Gauss, Johann Friedrich Carl, 43n, 

Gaussian densily function: 
conditional, 142, 128 
marginal, 125, 128 
mean value of, $1, 124 
N-dimensional (N-vuriate), 127, 128 
of one random vuriable, 43, 60, 61, 335 
two-dimensional (bivariate), 112, 124, 335 
vuriunce of, 81, 124, 336 

Gaussian distribution function: 
of one random varinble, 44, 335 
table of, 315 

Gaussian filler, 241 

Gaussian random process, 156, 158-160 
variance estimate of, 167 

Gaussian random variable(s): 
churacteristic function of, 83, 138 
conditional density function of, 128 
defined, 43, 124, 127 
linear transformation of, 77, 130-133 
marginal densily function of, 125, 128 
mean value of, 81, £24 
moment generating function of, 83 


Gaussian random variable(s): 
properties of, 128 
viriance of, Kf, 124, 336 


Idealized system, 209--250 
Iinpossible event, 10 
Tmiputse function: 
defined, 41, 310-312 
limiting forms of, 176, 195 
N-dimensional, 311 
relationship to unit-step function, 41, 311, 312 
two-dimensional, 311-312 
Impulse response of linear system, 206, 207 
Incomplete beta function, 33! 
Incomplete gamma function, 330, 331 
Incremental available power, 227 
Incremental noise power, 227 
Independent events, 18-22 
Independent experiments, 25 
Independent random processes, 148 
Independent random variables, 100-102, 
133-134 
joint characteristic function of, 122-124, $38 
Infinite set, 3 
Integrals: 
Fresnel, 273 
table of definite, 318-319 
table of indefinite, 316-318 
Integrator, 157 
Intersection of sets, 6 
Interval conditioning, 99 
Inverse Fourier transform (see Fourier transform) 


j, 60n. 
Jacobi, Karl Gustav Jakob, 1292, 
Jacobian, 129 
Joint central moments, 120-122 
of two discrete random variables, 136 
Joint characteristic function, 122-124 
of independent random variables, 138 
of two gaussian random variables, 83, 138 
Joint event, 89 
Joint moments, 119-120 
of two discrete random variables, 136 
Joint probability, 13 
Joint probability density function (see Probabil- 
ity density function) 
Joint probubility distribution function (see Prob- 
ability distribution function) 
Joint sample space, 88 
Jointly ergodic random processes, 152, 1571. 
Jointly gaussian random processes, 156, 160 
Jointly wide-sense stationary random processes, 
150, 160 -161, 213 


Khinchin, AJL, 180 


Faplace, Marquis Pierre Sinmon de, 39a. 
Laplace deasity function, 89, 6 
Laptice transform, (73 
Lethaia, CGiotthivd Wilhetn vou, Von 
feabaiz's cule, /6 
Likehhood, 9 
Linear system: 
causal, 210 
cross-correlation function, 213-214 
cross-power spectrums, 217 
detined, 206 
idealized, 209-210 
impulse response, 206, 207, 210 
noise temperature, 239 
optimum, 257 
output autocorrelition function, 212-213 
output mean-squared vitlue, 212 
output mean value, 211 
outpul power, 212, 216 
oulpul power spectrum, 215 
physically realizable, 210 
response to delerministi¢ signil, 200-209 
response to random signal, 211 
stable, 210 
lime-invariant, 207 
transfee function, 208 
white noise evaluation of, 214-215 
Liquid helium, 200 
Log-normal density finction, 62, 86, 336 
Log-normal distribution function, 62, 336 


Mapping, 35 
Marcum's Q-function, 305, 331 
Marginal characteristic function, 23 
Marginal density function (see Probability 
density function) 
Marginal distribution function (see Probability 
distribution function) 
Maser, 188rt., 200 
Matched filter, 258 
for colored noise, 259 260 
impulse response, 260, 261, 267 
maximum signal-to-noise ratio of, 260, 268 
oulput signal (rom, 268 
for rectangular pulse, 261-262 
Iransfer function, 260 
for white noise, 260-262 
Matrix, covariance, $27 
Mean frequency, 178 
Mean (expected) value: 
from awocorrelation function, 153 
conditional, 69, 139 
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Mean (expected) value: 
estimate of, 107 
of function of a random varieble, 68, 632 
of function of several randoa variables, 
{t7 bY 
of process sample function, 1M1 
ef causeclare processes} Plt 2st bed 
of random process derivitive, 166 
of random variable, 67-08, 133 
of sum of random variables, E18 
of system response, 211 . 
Meanesquared error, 263-266 
Mean time between failures (MTEL), 64 
Measurement of correlation functions, 156-158 
Median of random variable, 61 
Mixed random variable, 37 
Mode of random vacinble, 61 
Modulation index, 306 
Moment generating function, 73 
of gaussian random variably, 83 
Moments: 
central, 70, 82, 120 
from characteristic function, 72, 123 124 
interrelationships, 82 
joint, 119-122 
from moment generating function, 73 
about origin, 09, 82, 119-120 
of two diserete randouy variables, 136 
Monotonic transformations: 
decreasing, 78, 76 
increasing, 75-76 
Mutually exclusive events, 9 
Mutually exclusive sets, 3 


N-order stationary random process, 154 
Narrowband gaussian process, 220 
Narrowband random process, 220 
Natural frequency, 295 
Natural numbers, 80 
Noise, 1 

arbitrary source of, 228 

colored, 190) 

in controt system, 285-288 

in phase-locked loup, 288-295 

resistive source of, 227 

thermal, [K7, 227 

types of, 226n, 

white, 187 
Noise bandwidth, 210, 217-218, 238 
Noise figure: 

of attenuator, 238 

average, 236 

average operating, 236 

average standard, 246 
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Noise figure: Power: Probability density function: Random process(es): 
average sysiem operating, 40 in a random process, (63, 173,175 normal, 43n, autocorrelation function, 149, [52 184, 160 
incremental, Ad in tesponse al linear system, 212, 216 Paseal, 32 autocoviriance function, 155, 160 
operating spol, 235, 249 from second moment of process, 175 Poisson, 47, 8S, 332 band-limited, 219 
spot, 234 245 Power densily (see Power density spectrum) 7 properlics of 42, 53, 94 bandpass, 219 
standard spot, 235, 250 Power density spectrum, 153, 173 of random process(es), 148, classified, 144-146 
Noise performance: bandwidth of, 177-179 Rayleigh, 50, 336 complex, 160-161 
of amplitude modulation system, 278-279 of complex process, 193 | Rice, 299, 337 continuous, 145 
of frequency modulation system, 281-284 defined, 178 of sum of random variables, 102-105 cross-correlation function, 150, 184, 160-161 
of phase-locked loop, 292 295 : of derivative of process, (77 | transformation of, 74-80, 128-133 crass-covarinnce function, 156, 160-164 
of radar, 301-305 inverse Fourier transform of, 177, E80 | triangular, 4) ; cross-power specirutn, 182-185, 193 
Noise temperature: of output of tinear system, 215 of two random variables, 93 defined, 142, 160 
of antenna, 241 of product device output, 191 uniform, 48,337 density function of, $48 
of atienaaten, 24S Properties of, 176177 Weibull, RS, A AK derivative of, 166, 246 
average, be US telitionship to autocorrelation functioo, 174, Meababitity distiibution fonction: deterministic, 47 
average elective inpul, 247 179-181 AT CSHCS 333 diserele, 145 
average elective sauree, 237 relationship lo autocovariance function, 196 Bernoulli, 331 distribution function of, 147=148 
elective, 228 { Power gain, available, 232 heta, 333 ergodic, | $2 
effective input, 233 Power spectral density, (73a. binomial, 46, 332 estimate of, 168 
standard, 248 Power spectrum (see Power density spectrum) Cauchy, 62, 333 first-order stationary, 148-149 
system, 239 Power transfer function, 215 chi-square, 334 gaussian, 156, 18-160 
Nondeterministie randens process, 147 Prediction filler, 263 conditional, 52, 62, 96-99 jointly ergodic, 152, 157m. 
Nonstationary cundom process, 147 Probability, 9 defined, 37, 89, 147-148 jointly wide-sense stationary, 150, 160-161, 
Normal density function (see Gaussian densily u posteriori, 18 of discrete random varinble(s), 38, 90 213 
function) a priori, [8 Erlang, 34 mean of, 143, 153, 160 
Normat distribution fanction (see Gaussian dis. axioms of, £0 exponentinl, 49, 334 mean value estimate of, 167 
tribution function) conditional, 13 gamma, 335 N-order stationary, 151 
F Normalized second-order moment, 12! detined, 10 RAUSSIAN, 43-45, 345 narrowband, 220 
p Null set, 3 of detection, 102, 104-305 joint, 89, 90, 148 nondeterministic, 147 
2 of false alarm, 302, 303 taplnces 236 nonstationary, 147 
i : Open-loop transfer function, 285 joint, 13 log-normal, 62, 336 orthogonal, 154, 161 
Bat Operating spot noise figure, 235, 249 of miss, 302 Wah aise 92,93 periodic component in, 153 
: Operator, 206 ; total, 15 of one random variable, 37 power of, 13, 173, 175 ' 
rt Order of moment, 119, 120 transition, 1& Seg 332 random telegraph, 169 
[i ' Orthogonal random process, 154, 161 Probability density firnetion: Pos Sons 47, 332 sample function af, 143 
er Orthogonal random variables, 119, 134 aresine, 84, 339 cian of, 37-38, 91 second-order stationary, 149 
; Outcome of experiment, & Bernoulli, 431 of random process, 1470, 147-148 senirandom binary, 162, 296 
op Overmodulation, 2781., 306 beta, 333 Ray lelene Saat stationary, 147 
fit: binomial, 46, 81, 332 ean ; statistically independent, 148 
yet Parsevah Me Ae 3230. Cauchy, ot, M33 : several random variables, 90 stricl-sense stationary, ISf 
; Parseval’s theorem, 323 chi-square, 85, 331, 334 ‘ of sum of random variables, 102-105 time correlation functions of, 151-182 ue 
sie Pascal, Blaise, 132n. conditional, 52, 96-99 of ee et faa uncorrelated, 156, 161 
4 Pasgal density, 332 delined, 40, 93, 148 Waa 48, 337 variance of, 156 
in Phase detector, 290 of discrete random variable(s}, 41, 94 iacate . wide-sense stationary, 150, 160-161 
Point conditioning, 96-99 Erlang, 334 | ria s pie anaes pee Random sequence: ki 
i Poisson, Simean Denis, in existence, 1 H toduct device, 191 193 continues, 146 
t Penance ‘ ’ spottential fh, Ud BR ENCE UAL SEES. Le discrete, FG 
Torvvseons afearsaty Pratt Gan, be SS EY au ptertte tie : discudiel Site Awards ional, 88 ; M " 
I: pietatian Mt ake Bunn 308 ae 1 ace, Iword imensional, Random signal, | " 
suiriinee of, SE, 332 gaussian, 14,60, G1, 112, 124 $28, MS ulse repetition frequency (PRE), 301 Random telegraph process, [69 i: 
Paisson random variable, 47, 140 joint, 93 Random variable(s): | 
Power: Laplace, 99. 81, 316 Quantizer, $8 auxiliary, 140 
from antocerrelition function, 153 logenormal, 62, 86, 446 j ho al i 
incremenLeb available, 227 marginal, 9S, % i : F central moments of, 70, 82, 120-122 ‘ 
5 from power density spectrum, 175 of N random variables, 94 Random point, 88 characteristic function of, 72, 122 ; 
; I 
mS ' 
‘ | 
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Randont variable(s): 
coellicient of skewness, 71 
complex, 133 
conditions for, 36 
continuous, 37 
correlation of, 119 
correlation coefficient, 124, 136 
covariance of, $21, 134 
delined, 34, 88, 89 
discrete, 37 
Erlang, 85 
gaussian, 43, 81, 1197, 124-128, 130-133 
independent, 100-102, 133-134 
joint central moments of, 120 
joint characteristic function of, 122-124, 138 
joint moments of, 119, 123, 124 
lattice-type, 106n. 
marginal characteristic functions of, 123 
mean of, 67-68, 133 
median of, 61 
mixed, 37 
mode of, 61 
moments of, 69--70 
N-dimensional, 89 
orthogonal, 119, 134 
Poisson, 47, 140 
Rayleigh, 50, 81 
skew, 7] 
standard deviation, 70 
transformation of, 74-80, 128-130, 130-133 
uncorrelated, 119, 134 
uniform, 48, $9 
variance of, 70, 133 
vector, 88 
Weibull, 85 
Random vector, 88, 89 
Range, 58 
Range sumple spuce, 88 
Rational power spectrums, 267 
Rayleigh, Lord John William Strutt), 50a. 
Rayleigh density function, 50, 336 
maximum value of, 61 
mean of, 81 
median of, 61 
mode of, 61 
variance of, 81 
Rayleigh distribution function, 50, 337 
Ruyleigh random variable, 61 
Realizable linear system, 210 
Realization of a process, 143 
Rice density, 337 
rms bandwidth, 177-179, 198, 201 


Sample function, 143 


Sample space, 9 
combined, 22 
continuous, 9 
discrete, 9 
joint, 88 
range, 88 
Sampling function, 326 
Schwarz, Hermann Amandus, 259n. 


Schwarz's inequalily, 259 
Second-order stationary random process, 149 
Semirandom binary process, 162, 296 


Series, table of, 319 


Set(s): 


algebra of, 7 

associative law of, 7 

class of, 3 

commutative law of, 7 

complement of, 6 

countable, 3 

countably infinite, 3 

defined, 3 

difference of, 5 

disjoint, 3 

distributive law of, 7 

elements of, 3 

empty, 3 

equality of, 5 

finite, 3 

infinite, 3 

intersection of, 6 

mutually exclusive, 3 

null, 3 

product of, 6 

sum of, 6 

uncountable, 3 

union of, 6 

universal, 4 

Venn diagram of, 5 
Sct theory, 2 
Skew, 70 
Skewness, 70 
Smoothing filter, 263 
Spectrum (Fourier transform), 72, 173 
Spot noise figure, 234-235, 249, 220 
Stable linear system, 210 
Standard deviation, 70 
Standard noise temperature, 235 
Standurd source, 235 
Stationarily (see Random process) 
Statistical average (see Expected vulue) 
Statistical independence: 

of events, 18-22 

of experiments, 25 

by puirs, 19 
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Statistical independence: ; 
of random processes, [48 
of random variables, (00-102, 134 
Subexperiments, 22 
Subset, 3 
Sums of random variables: 
density function of, 102-105 
distribution function of, 102-105 
mean of, LIK 
variance of, 122 
System noise temperature, 239 


Tables: 
of definite integrals, 318 
of finite series, 319 
of Fourier transform pairs, 329 
of gaussian distribution function, 314 
of indefinite integrals, 316-318 
of trigonometric identilies, 315-316 
Telegraph process, 169 
Thermal noise, 187 
Thevenin, Léon, 2270. 
Thevenin voltage source, 227 
Threshold, 284, 302, 303 
Time autocorrelation function, (St 
Time uverage, 15! 
Time cross-correlation function, 152 
Time-domitin anitlysis, 172 
Time-invariaat system, 207 
Total probability, 15 
Transfer function: 
open-loop, 285 
of phase-locked loop, 290-292 
of system, 208, 215 


Transformation of random variable(s), 74--80, 


128-130 
linear, 77, 130-133 
monotonic, 75-77 
nonmonotonic, 77-80 
square-law, 78-79 
Transition probability, 18 
Triat, ¥ 
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Togonometiic identities, bible of, TS S16 
‘Two-danensional product space, 88 


Uncorrelated random processes, 156, 161 
Uneorrelated random vieribles, (19, 134 
Uneountable set, 3 
Uniform density function, 48, 139, 337 
mean of, 80, 337 
varianee af, BO, 337 
Union of sets, 6 
Unit-inipulse function (see Impulse function) 
Unit-step function: : 
defined, 38, 314, 312 
related to impulse function, 311, 312 
two-dimensional, 311-342 
Universal set, 4 


Variance: 
estimate of, 167 
of random pracess, 156 
of random variable, 70, 133 
of sum of random variables, 122 
Vector candom variable, 8X 
Venn, John, Sa. 
Venn diagram, 5 
Voltage density spectrum, 173 


Weibull density, 85, 338 
Weibull, Erost EH, W., 85a. 
White noise: 
autocorrelation function of, [87 
band-timited, 188 : 
Uefined, 187 
in system evaluation, 214-215 
power spectrum of, 187 
Wide-sense stationary random process, 150, 
160-161 
Wiener, Norbert, 180, 262, 
Wiener filter, 262-266, 288, 307 
minimum mean-squared error of, 266 
teansfer function of, 265 : 
Wiener-Khinchin relations, 180 


